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ABSTRACT 

The first of four issues in this volume consists of 
four review articles. "Tech Prep Quality" reviews a guide to total 
quality management and tech prep. "Specific, Ocean to Ocean" reviews 
a publication that identifies 10 preliminary plans for implementing 
tech prep. "...No Single Option" is about a monograph that evaluates 
tech prep and other current initiatives in terms of their value to 
the middle 50 percent of students. "Tech Prep Nat Sat Telecon" 
reports on a satellite teleconference that explored leadership, 
marketing, and business involvement in tech prep. Issue 2 begins with 
"Authentic Assessment," a review of authentic assessment practices. 
"Collecting Alternative Approaches to Assessment" focuses on 
alternative systems of assessment that address reforni directives and 
new approaches to vocational education. "Let Me Cet This Straight" 
^ooks at study results of local institutions who are trying to 
understand what Perkins accountability mandates really mean. 
"Accountability" summarizes two National Center for Research in 
Vocational Education (NCRVE) reports --one that develops a model of 
accountability and another that proposes policy alternatives in 
response to federal legislation. Issue 3 contains four articles. "An 
Annual Report on School- to -Work Programs" reviews an NCRVE report of 
the research conducted on successful education- to -work transition 
approaches. "Dive Right In!" explores work-based learning for medical 
unit secretaries. "Special Populations and More" reviews publications 
addressing two school- to-work connecting activities areas- -special 
populations and gender equity. "Revising Educational Assessment" 
reviews two items about accountability and assessment. Issue 4 begins 
with "The Right Instrument," a review of the guides to two tools to 
assess leadership performance. "Administrative Breakwater" reviews a 
professional development simulation. "In Cases..." focuses on a 
collection of case studies for leadership development. "For Those 
about To Lead" reviews a leadership development curriculum. "A Source 
of Resources" assesses a publication that provides detailed 
information about leadership development resources. (YLB) 
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ABOUT NCR VE 



NCRVE is established under authorization of the Carl D. 
Perkins Vocational Education Act to engage in two 
vocational education-related activities: (1) applied 
research and development and (2) dissemination and 
training. Our mission is to address the goals of preparing 
individuals for substantial and rewarding employment 
and of acting as a catalyst for a shift to an economy 
dominated by a skilled and flexible workforce. NCRVE is 
a consortium of educational institutions led by the 
University of California at Berkeley's Graduate School of 
Education. This consortium also consists of the Teachers 
College at Columbia University, the University of Illinois, 
the University of Minnesota, RAND, Virginia Polytechnic 
Institute and State University, and the University of 
Wisconsin at Madison. In addition to its research and 
development agenda, NCRVE supports five dissem- 
ination and training programs: (1) dissemination; (2) 
professional development; (3) special populations; (4) 
planning, evaluation, and accountability; and ( n >) program 
development, curriculum, and instructional materials. 
This publication is part of our commitment to those on 
the front lines of vocational education, striving to create 
a new vision. 
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TECH PREP QUALITY 

A TQM Approach to Tech Prep 



Tech Prep is a burgeoning educational reform 
initiative — perhaps the most significant 
secondary and postsecondary educational 
initiative to develop in the United States in recent 
years. 

Implementing Tech Prep: A Guide to Planning a 
Quality Initiative by Debra D. Bragg provides 
perspectives on Tech Prep from personnel 
representing state agencies and many local consortia 
in each of the fifty states. Many of the individuals 
whose perspectives are collected in Implementing 
Tech Prep have been involved in the initiative since 
before the passage of Perkins II, the Carl D. Perkins 
Vocational and Applied Technology Act 
Amendments of 1990, a part of which is the Tech 
Prep Education Act. 

As the introduction to the guide points out, 
establishment of the Tech Prep initiatives of Perkins 
II is an effort to confront a myriad of serious and 
fundamental educational and economic concerns, 
including the following: 

° Schooling lacks meaning for a large proportion of 
high school students, often referred to as the 
neglected majority. 

• America's educational system fails to efficiently 
transition many young people from high school to 
work or to further education. 

° Technically demanding occupations are growing 
at a faster rate than America's technically skilled 
workforce. 

• The United States is competing in a global 
marketplace where foreign competitors' workers 
are more highly trained and productive than its 
own. 

'Tech Prep is seen as a potential solution to these 
problems," Bragg and other proponents of the 
initiative contend, "because of its comprehensive, 
applied, and technologically-rich approach to 
blending secondary and postsecondary education for 
employment." 



Implementing Tech Prep is a practical guide, first, 
because many of the ideas in it have their source in 
successful examples of Tech Prep implementation 
throughout the country and in a variety of other 
initiatives in education as well as business and 
industry. Consequently, Implementing Tech Prep 
presents a potential approach to Tech Prep that has 
the fundamentals of Total Quality Management 
(TQM), an initially business-oriented but now 
expanding initiative, as its foundation. At the same 
time, however, contributors to the guide claim that 
the guide "is not a cookbook advocating only one 
approach. Rather, it considers new ways of thinking 
about the educational innovation promised by Tech 
Prep" — prominent among them, an approach based in 
the similar TQM initiative. Implementing Tech Prep 
considers first the nature of traditional educational 
reform approaches and Tech Prep's relation to them 
then explores the similar principles and procedures 

The Total Quality 
Management approach to 
implementing Tech Prep 
seems a reasonable and 
well-founded one 

associated with both Tech Prep and TQM, showing 
how the two initiatives complement one another in 
developing comprehensive Tech Prep programs. 

Implementing Tech Prep is a practical guide not only 
for its content, but also for its format. The guide was 
written using techniques of the Information Mapping® 
method, a writing system that employs the use of 
essentially self-contained, modular units and blocks 
of text that, as the guide indicates, 

enable a reader to choose information that is 
needed without having to sort through pages of 
text. . . . And, because it is easy to find 
information and move through the document in 
ways that make sense to individual readers, it is 
also easier to learn and relearn information. 
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Reading time for documents written using 
Information Mapping®, the guide reports, is reduced 
by as much ?s 40%. 

Hence, Implementing Tech Prep presents a great deal 
of information about planning and implementing 
Tech Prep and presents that information in an easy- 
to-use format: "I've received very positive comments 
about the formatting of the document from 
practitioners," Bragg reports. 'They think it is quick 
2Jid easy to use." 

After an introductory chapter, Implementing Tech 
Prep begins by providing a historical perspective for 
Tech Prep initiatives. Tech Prep, according to Bragg 
and chapter coauthor Carolyn J. Domsife, has its 
origins in a long-standing debate among educators 
and policymakers that addresses two questions: 

1 . What is the appropriate role and function of job 
training and vocational education as part of the 
nation's public education system? 

2. What are the educational consequences of 
technological change in the workplace? 

Implementing Tech Prep traces the concept of Tech 
Prep, its variations, and its successes from legislation 
such as The Manpower Development and Training 
Act of 1962, through the reform strategies of the 
1980s, and beyond Perkins II, which in 1990 formally 
defined Tech Prep as an education program that 

• leads lo an associate degree or two-year 
certificate 

• provides technical preparation in at least one field 

• builds student competence in mathematics, 
science, and communication through sequenced 
course work 

• leads to placement in employment 

In the 1990s, schools are struggling to better educate 
more students by using resources more effectively. 
Though one of the more ambitious, Tech Prep is one 
among several reform options. In Implementing Tech 
Prep's, third chapter, Dornsife discusses the need for 
accepting the reality of reform; Tech Prep and the 
four aspects of educational restructuring (curriculum 
and instruction, authority and decision making, staff 



roles, and accountability); and the challenges to and 
federal support for Tech Prep implementation, 

Chapter Four of Implementing Tech Prep, "Initiation 
of Tech Prep by the Fifty States," is based around a 
study conducted by NCRVE to document the 
initiation of Tech Prep in the states by local consortia. 
Like the study itself, Bragg and James D. Layton 

This is a practical guide 
not only for its content, 
but also for its 
easy-to-use format 

concern themselves in the chapter with the state grant 
funding of the consortia that became effective with 
Perkins II in July 1991 as well as the predominant 
philosophies, policies, and components of initial state 
Tech Prep initiatives. After reviewing the study, 
Bragg and Layton draw some implications from its 
findings — specifically, implications about resources 
(time, staff, and money), guiding philosophy, 
program evaluation, and the role of community 
colleges. "Whether [Tech Prep] will ultimately 
succeed or fail depends on the degree to which 
current problems are solved and surmounted," the 
authors conclude. 

Another study conducted by NCRVE personnel, this 
one as part of the Implementing Tech Prep project, 
informs the "Planning & Implementation of Tech 
Prep by Local Consortia" chapter of Implementing 
Tech Prep, which describes how individual 
organizations (mostly consortia and community 
college servhe regions) have begun to plan and 
implement Tech Prep. The chapter discusses aspects 
of Tech Prep planning and implementation such as 
definition and strategy setting, consensus building, 
selection and orientation of planners, consortium- 
level planning, wide-scale orientation and 
recruitment, and initial implementation. Identified by 
the study were a number of critical purpose- and 
process-oriented issues, including clarity of purpose, 
high school tracking, vocational education control of 
administration and funding, resource needs, 
leadership development, flexibility in implementation 
and evaluation, establishing meaningful partnerships, 
and maintaining momentum. Each issue is discussed 
in detail in the chapter. 
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It is at this point in Implementing Tech Prep that a 
discussion of Total Quality Management (TQM) is 
integrated with the discussions of Tech Prep. The 
contributors' integration of TQM and Tech Prep are 
based largely on parallels between the two: reforming 
and improving systems, empowering teams 
representing a variety of stakeholders to make 
improvements, and using measurements as the basis 
for all.continuous quality improvement efforts. The 
sixth chapter of the guide explores TQM principles; 
quality theories; and finally, implications of TQM for 
Tech Prep implementation in the areas of leadership, 
vision, goals, resources and training, measurement, 
performance review, rewards for quality 
improvement, and the challenges of applying TQM to 
Tech Prep. The challenges for the TQM-Tech Prep 
synthesis Bragg explores are extrapolated from the 
difficulties other business and educational institutions 
have encountered in applying TQM in their 
situations, though she warns that "it is impossible to 
predict eveiy challenge that could stand in the way of 
using TQM to implement Tech Prep." 

Catherine L. Kirby and James D. Layton's brief 
"Implementing Tech Prep with Teams" chapter 
attempts to reinforce the need for effectively 
employing teams to accomplish the Tech Prep 

Tech Prep is perhaps the 

most significant 
educational initiative in 
the United States in 
recent years 

mandates of Perkins II and provides a transition into 
the more extensive eighth chapter, "How to Use 
Group Processes and Quality Tools." While Chapter 
8 is intended as a mere introduction to some of the 
techniques and tools used in TQM team funct ns, 
Layton refers readers to a set of additional resources. 
The references provide the details of group 
techniques and quality tools of which those listed in 
the chapter— multivoting, action planning, 
benchmarking, and computer-assisted group 
techniques among them — are but a few examples. 

The final chapter of Implementing Tech Prep 
summarizes the guide and provides an outline of an 



approach to implementing Tech Prep by applying 
eight essential prerequisites for strategic quality 
management: 

1 . gaining support from leaders 

2. creating a shared quality vision and policies 

3. formulating and prioritizing quality goals 

4. deploying goals throughout a consortium 

5. providing necessary resources, including training 

6. stating desired outcomes and establishing 
measures 

7. reviewing performance regularly 

8. revising the reward system 

Treatment of each prerequisite includes goals, 
planning activities, team roles, tools, and outcomes. 

In addition to summarizing the fifty-one-state survey 
results and acknowledging the NCRVE Tech Prep 
research sites, the appendices of Implementing Tech 
Prep briefly outline steps for successful articulation 
of students within Tech Prep programs. 

The Total Quality Management approach to 
implemcr :ing Tech Prep established in Implementing 
Tech Prep: A Guide to Planning a Quality Initiative 
seems a reasonable and well-founded one. The 
synthesis of these two innovative initiatives — as well 
as the Information Mapping® format in which it is 
presented — deserves consideration by educational 
systems responding to the Tech Prep mandates of 
Perkins II. Final decisions about how to implement 
Tech Prep remain, of course, with state agencies and 
local consortia. 



Implementing Tech Prep: A Guide to Planning a Quality 
Initiative (MDS-241 ) was prepared by Debra D. Bragg 
fiom the NCRVE site at the University of Illinois at 
Urbana-Champaign. (For ordering information, see page 
12.) 
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SPECIFIC, OCEAN TO OCEAN 
Ten Plans for Integrated Tech Prep 



rhe excerpt belo*v is from the preface of the 
brand new Establishing Integrated Tech Prep 
Programs in Urban Schools: Plans Developed 
at the NCRVE 1993 National Institute, one in a series 
ofNCRVE Technical Reports. As the excerpt 
indicates, Establishing Integrated Tech Prep contains 
ten preliminary plans for implementing Tech Prep in 
high schools and affiliated community colleges in ten 
U.S. cities: Washington, DC; Cleveland, Ohio; Las 
Cruces, New Mexico; St. Paul, Minnesota; 
Philadelphia, Pennsylvania; Tuscaloosa, Alabama; 
Detroit, Michigan: Oklahoma City, Oklahoma; 
Raleigh, North Carolina; and Baltimore, Maryland. 

As a nearly 500-page anthology of comprehensively 
devised plans, Establishing Integrated Tech Prep 
provides Integrated Tech Prep program planners 
with a tremendous "volume rr of specifics. 

The National Center for Research in Vocational 
Education (NCRVE) held its third National Institute, 
"Establishing Integrated Tech Prep Programs in 
Urban Schools," in Berkeley, California, July 14-23, 
1993. Ten teams of vocational and academic 
educators — 125 high school and community college 
teachers, counselors, and administrators — from ten 
major metropolitan areas, selected through a 
competitive application process, participated in this 
institute. Each team was aided by a representative of 
its state department of education. Several teams were 
also assisted by local business partne .. Also working 
with each team were two mentors — an expert 
practitioner and a graduate student. Speakers, panel 
discussions, breakout sessions, and role play 
exercises were all part of the institute experience. In 
the course of the intensive two week program, each 
team was charged with the responsibility of 
developing a strategic plan for initiating or continuing 
the development of an Integrated 1 ech Prep program 
within its institutions. 

The Carl D. Perkins Vocational and Applied 
Technology Act of 1990 (Perkins II) mandated the 
integration of vocational and academic education as 
prerequisite for other funding, including funding 
through the Tech Prep Education Act. By enhancing 
academic competencies in vocational programs and 



making academic curricula more accessible to all 
students, the integration of vocational and academic 
education has become a vital component of all 
successful Tech Prep initiatives. Strong Tech Prep 
initiatives can make powerful contributions to 
building student competence in academic areas and 
vocational-technical fields. Through collaborative, 
articulated efforts between secondary and 
postsecondary institutions, these programs smooth the 
transition from school to work by providing students 
with high quality workplace skills. 

Perkins II also addressed the needs of disadvantaged 
youth, especially those in urban areas, by targeting 
funding to areas of poverty and special needs. As a 
result, many large cities have received substantially 

If the Integrated Tech 
Prep planning effort is to 
be successful, it must 
effectively involve the 
entire community 

increased federal funding. Unfortunately, many urban 
recipients lack the technical information to plan and 
implement successful programs. Developing 
workable strategies to create better schools for 
students requires an understanding of the issues; a 
commitment from the leaders; and the support of 
superintendents, administrators, teachers, and 
business. Creating such strategies was a major goal of 
the 1993 Institute. 

The 1993 Institute built upon and was strongly 
influenced by [NCRVE's] work with twenty urban 
school teams who took part in NCRVE's first two 
Institutes in the summer of 1992 and with whom 
[NCRVE] has developed its Urban Schools Network. 
Monitoring site visits, discussion at regional 
meetings, progress ~eports, and end-of-year 
evaluations frc m these teams helped [NCRVE] to 
refine . . . , re'.bcus, and reshape the 1993 Institute. 
The active participation and ample feedback provided 
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by the participants in [the] Urban Schools Network 
was vital to its success. The 1993 Institute 
participants now join the Urban Schools Network, 
which encourages exchanges of information between 
those with extensive experience in various program 
areas and others just beginning to learn about them. 
During the year, NCRVE sponsors regional meetings 
at which network teams can visit model programs, 
assess their own progress, and plan future steps. 
[NCRVE* s] ultimate goal is the development of a 
practitioner "owned" reform movement within 
vocational education, one that is organized in school 
and communities but coordinated and supported on a 
national level. 

[Establishing Integrated Tech Prep] is comprised of 
ten planning documents developed at the 1993 
Institute. It is important to note that these planning 
documents represent a culminating activity of the 
1993 Institute and are not the final Integrated Tech 
Prep plans for any of the institutions involved. 
[NCRVE is] acutely aware that if the Integrated Tech 
Prep planning effort is to be successful, it must 
effectively involve and engage the entire community 



involved with Tech Prep and that the effort must be 
an ongoing, incremental process [that] will continue 
long after the institute is over. [NCRVE] trusts that 
these plans will form a substantive basis for the 
required consultation process which will take place in 
the coming months. 

[NCRVE' s] intent in publishing these documents is to 
assist other community college and high school 
planners who are initiating or enhancing their own 
Integrated Tech Prep programs by providing specific 
examples of plans designed for urban areas. [NCRVE 
is] pleased by the high quality of the ideas set forth in 
these plans and are proud to have been associated 
with the teams which produced them. 

Establishing Integrated Tech Prep Programs in Urban 
Schools: Plans Developed at the NCRVE J993 National 
Institute (MDS-770) was compiled at the NCRVE site at 
the University of California at Berkeley. (For ordering 
information, see page 12.) 



. - - NO SINGLE OPTION 

Assessing Tech Prep and Other Educational Reforms 
on Behalf of the "Excluded Middle" 



The authors of Building the Middle- -Sue E. 
Berryman, Erwin Flaxman, and Morton 
Inger — bill their monograph as an 
"interpretive synthesis" of NCRVE research about 
educational reforms concerning the preparation of 
students for postsecondary training and work: 

Work preparation traditionally has been thought 
of as the venue cf students not able to do rigorous 
academic work — those not bound for college — 
but we have learned that the cognitive principles 
of work-related education are the foundation for 
the learning necessary for both further academic 
education and success in a changing workplace at 
all job levels. We strongly feel that the nation 
needs to objectively examine the various 
proposals for work preparation, some already 
implemented, to build an educatively powerful 
training system for students, not just a patchwork 



of different programs, not one of which meets the 
basic criteria for positioning students for middle- 
skill jobs. 

While the authors acknowledge the contributions of 
non-NCRVE research on a broad work preparation 
system and urge readers to consider their monograph 
alongside such research, they intend for Building the 
Middle to present NCRVE research as "a coherent 
and unified body of knowledge" capable of 
contributing to the design of a comprehensive, 
national work preparation system. The reform 
initiatives identified from NCRVE research for 
evaluation are cognitive apprenticeship, Tech Prep, 
integrated vocational and academic education, 
vocational education as part of general education, 
career magnet schools, academies, work-based youth 
apprenticeship, cooperative education, and school- 
based enterprise. 
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A comprehensive system of work preparation is the 
proposed solution to a two-pronged problem: On the 
one hand, the American economy does not reward 
low skills with middle-level wages as it once did; on 

No single initiative meets 

Building the Middle's 
proposed design criteria 

the other, the American education system — which 
"now takes seriously the education of only the top ten 
or twenty percent** of K-12 and postsecondary 
students — is "missing the middle," providing no 
coherent educational strategy for students who 
complete high school but are unable vo complete 
college or the training necessary to enter middle-level 
jobs. The proposed solution — building the middle — 
requires the involvement not only of schools, but of 
employers as well: it requires integrating "the 
demand and supply sides of the equation": 

Building the middle requires changing many 
pieces of the American education and training 
system — its curriculum; pedagogy; organization 
c< iime; assessment practices; technology; the 
skills and knowledge of its teachers; and the 
relationships among K-12, postsecondary 
education, and workplaces, and among academic 
teachers, vocational teachers, learners, and 
employers. 

Building (he Middle specifies seven design criteria for 
evaluating initiatives, identifies the defining 
characteristics of each initiative, clarifies the 
similarities and differences among the initiatives 
currently being considered, discusses the strengths 
and weaknesses of each initiative in meeting the 
proposed design criteria, and identifies barriers to the 
adoption and diffusion of each initiative. Coupled 
with the variety of reform initiatives considered, it is 
especially these analyses that make Building the 
Middle a valuable resource. 

The authors of Building (he Middle develop their 
seven design criteria with the overall educational 
objective of creating learning arrangements that are 
"so highly motivating and so effective that all 
students develop the knowledge and skills needed for 
at least middle-skill and middle-wage jobs.** The 
criteria define what the authors see as essential 



elements of a comprehensive work preparation 
system. An initiative contributes to the solution of 
building the middle, according to the authors, if it 
accomplishes the following: 

1. sets up educationally rich and problem-rich 
learning activities that reflect the knowledge 
demands of the work contexts in which 
knowledge and skills have to be used 

2. creates a "community of expert practice," where 
the adults do more than talk about the practices of 
that community and play the role of subject- 
matter coach during the learning process 

3. develops knowledge and skills efficiently 

4. engages employers in curricular design, the 
creation of work-experience positions, and 
commitments to hire the program's graduates 

5. makes the relationships and tradeoffs between 
different training investments transparent for 
students 

6. is designed to prevent its being captured by the 
more advantaged or abandoned to the less 
advantaged 

7. can potentially scale up into a national system 
that prepares students for middle-skill and 
middle-wage jobs 

The monograph's assessment of the reform options it 
considers is substantial and, for all practical purposes, 

The cognitive principles 
of work-related education 
are the foundation for the 
learning necessary for 
both further academic 
education and success in a 
changing workplace 

requires firsthand consideration on the part of 
educators, policymakers, and others interested in 
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developing a comprehensive, national solution to the 
problem of the "excluded middle." The monograph 
warns that building the middle "will require an 
intelligent and efficient infrastructure to support the 
process"; briefly, they propose infrastructure changes 
in the areas of quality assurance, curricula, 
professional development, and knowledge transfer. 

Ultimately, Building the Middle concludes that no 
single option meets the proposed design criteria. 
Instead, the monograph's authors propose a solution 
that integrates the pedagogic and curricular principles 
of cognitive apprenticeship, the organizational 
elements of vocational and academic integration, the 
articulation and occupational cluster principles of 



Tech Prep, and the rigorous skills certification of 
work-based apprenticeships. 

A brief overview of Building the Middle — Building 
the Middle: Executive Summary — is also available 
separately from the NCRVE Materials Distribution 
Service. 



Building the Middle (MDS-408) and Building the Middle: 
Executive Summary (MDS-409) were prepared by Sue E. 
Berryman, Erwin Flaxman, and Morton Inger from the 
NCRVE site at Teachers College, Columbia University. 
(For ordering information, see page 12.) 



TECH PREP NAT SAT TELECON 

Leadership, Marketing, and Business Involvement 

in Tech Prep ... via Satellite 



/nvolving Key Players in Tech Prep is a 
videocassette recording of one in a series of 
1992 NCRVE National Satellite Teleconfer- 
ences. The teleconference series, "Breaking the Mold: 
Tech Pi'ep and Integration of Academic and 
Vocational Education," explores the Tech Prep and 
integration of vocational and academic education 
reforms and includes three other videoconference 
recordings: Laying the Foundation for Integration, 
Building Teams for Tech Prep, and Achieving 
Integration Through Curriculum Development. 

The one and a half hour Involving Key Players 
teleconference, which took place on December 15, 
1992, was broadcast live via satellite from Virginia 
Polytechnic Institute and State University to 
participants at downlink sites across the United States 
and Canada. 

Involving Key Players opens with a short address by 
Dale Pamell, Profes \ in the School of Education at 
Oregon State University, autho r of the seminal book 
The Neglected Majority, and one of the founders of 
the Tech Prep movement. In his overview, Parnell 
notes the apparently significant impact of Tech Prep 
programs on teaching and learning. He also 
emphasizes the notions that Tech Prep integrates 
content with context, vocational education with 



academic education, and that the reforms give 
attention to the majority of students — who are 
unlikely to complete baccalaureate programs — and 
improves the match between what students need for 
work and other life roles and what they are taught in 
school. 

The teleconference was set up as a combination panel 
discussion and question call-in session. Joining 
moderator David Crippens of KCET-TV were 
panelists Julia B. Akers, Coordinator, Roanoke 
(Virginia) Area Tech Prep Consortium, and member 
of the Virginia Tech Prep Advisory Board; Stephen 
D. Spaar, Coordinator, Technical Division, T. H. 
Pickens Technical Center in Aurora, Colorado; 
Donald C. Garrison, President, Tri-County Technical 

Tech Prep integrates 
content with context, 
vocational education with 
academic education 

College in Pendleton, South Carolina; and Carver C. 
Gay ton, Corporate Director of College and University 
Relations, Boeing Company, Seattle, Washington. As 
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representatives of business and both secondary and 
postsecondary educators at various levels, the 
panelists brought a range of experience with the Tech 
Prep implementation process to the broadcast. 

The objective of the teleconference was to share 
strategies for involving key players in Tech Prep 

There are people in 
districts who can design 
Tech Prep initiatives 

that fit the needs 
of local communities 

initiatives. Specifically, the teleconference addressed 
three questions: 

1 . Why is leadership an issue in Tech Prep? 

2. How can Tech Prep programs be successfully 
marketed? 

3. How can business and industry become involved 
in Tech Prep programs? 

The balance of the first hour of the teleconference 
consisted mostly of panelists' reactions to three 
segments of prerecorded interviews with people in 
business and with administrators and faculty from 
various secondary and postsecondary institutions. In 
each segment — which addressed, respectively, the 
issues of leadership, marketing, and business/industry 
involvement — interviewees provided the details of 
their experience with Tech Prep and offered their 
opinions about the implementation and efficacy of 
Tech Prep programs. 

The call-in questions during the first hour set the 
stage for involving participants at downlink sites 
more fully later in the broadcast. 

Most of the last half hour of the teleconference was 
devoted to questions from viewers at the various 
down-link sites. Questions addressed the following 
issues: 

• ways in which special populations are served by 
Tech Prep 



« roles of industrial technology staff 

• articulation of Tech Prep students to adult 
vocational centers 

• disinvestments necessary for implementation of 
Tech Prep 

• effects of Tech Prep on tracking 

• teacher strategies for cultivating leadership, 
marketing programs, and involving business and 
industry for Tech Prep programs 

• small business involvement in Tech Prep 

• proven methods for involving business in Tech 
Prep 

• role of the Principles of Technology and other 
applied academics programs in Tech Prep 

• overcoming the mindset of having students 
complete four-year degrees 

o preventing schools and consortia from 
whitewashing existing programs rather than 
establishing substantial Tech Prep programs 

• employment of Tech Prep program completers 

• place of humanities and social sciences in Tech 
Prep 

• preparing students for college entrance 
examinations and in terms of postsecondary 
admission standards 

• parent resistance to Tech Prep 

• establishing Tech Prep competencies and 
performance standards 

• matching the requirements of and credits awarded 
for Tech Prep programs with four-year college 
and university admissions requirements 

• contacting businesses about their involvement in 
Tech Prep 

• involving community college faculty in Tech 
Prep 
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• changes in technical core areas as a result of Tech 
Prep 

• expectations of student performance 

• the entrenched negative image of vocational 
education and vocational programs 

In general, the questions and panelists responses 
touched on a wide range of significant issues — most 
notably, many of the pivotal issues that proponents of 

Tech Prep improves the 

match between what 
students need for work 
and other life roles and 
what tthey are taught 

Tech Prep have addressed and continue to address 
during the early years of the initiative's 
implementation. 

Before closing the teleconference with a graphic 
review of some of the important points made during 
the teleconference, moderator Crippens asked the 
panelists to indicate the one thing they would or 
woulc- not do if they had their Tech Prep programs to 
do ovt »• again. Below are their responses: 

I would involve . . counselors more [in] the 
program than we did with the original 2+2 
[program]. 

— Julia B. Akers 

Communicating to . . . four-year institutions — I 
think we needed to do that. 

— Donald C. Garrison 

I think the major thing that we have to do ... is 
to ensure that there are other businesses, other 
industries that are involved with r^ 6 ard to the 
process. . . . [Many people] see this as an 
educational program almost exclusively. . . , You 
don't see . . . many of the businesses that should 
be involved. And really, we do have to think 
about equal partners working together to achieve 
[what] Tech Prep is all about. 

— Carver C. Gayton 



[Beyond not] involving the entire community, . . . 
I would not . . . be so intimidated by the [big Tech 
Prep programs] that have been so successful, and 
fall back more on the resources that [we] have in 
[our] own districts and realize that [we have] 
professional, educated people in [our] districts 
[who] can design a Tech Prep model or initiative 
that fits the needs of the local community. 

— Stephen D. Spaar 

NCRVE is committed to keeping educators up-to- 
date on Tech Prep and the integration of vocational 
and academic education. Additionally, NCRVE is 
interested in learning about exemplary programs that 
can be shared with others. If you know of an 
exemplary Tech Prep or integration program, please 
write to 

Laurie Hollana 
NCRVE 

University of California at Berkeley 
2150 Shattuck Avenue 

Suite 1250 
Berkeley, CA 94704 

or call (800) 762-4093. 

NCRVE is planning national satellite teleconferences 
for this year, but details are not yet available. For 
further information, call Susan Faulkner at (703) 231- 
7337 or Nora Hansen at (703) 23 1-5754, or contact 
them by e-mail at ncrve@vtvml.cc.vt.edu. Both 
Faulkner and Hansen are from the NCRVE site at 
Virginia Polytechnic Institute and State Univeisity. 

The videotape Involving Key Players in Tech Prep 
(MDS-467) was produced at the NCRVE site at Virginia 
Polytechnic Institute and State University. (For ordering 
information, see page 12.) 

The videotape laying the Foundation for Integration 
(MDS-464), Building Teams for Tech Prep (MDS-465), 
and Achieving Integration Through Curriculum 
Development (MDS-466) were also produced at the 
NCRVE site at Virginia Polytechnic Institute and State 
University. (For ordering information, see page 12.) 
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NCRVE is established under authorization of the Car! D. 
Perkins Vocational Education Act to engage in two 
vocational education-related activities: (1) applied 
research and development and (2) dissemination and 
training. Our mission is to address the goals utf preparing 
individuals for substantial and rewarding employment 
and of acting as a catalyst for a shift to an economy 
dominated by a skilled and flexible workforce. NCRVE is 
a consortium of educational institutions led by the 
University of California at Berkeley's Graduate School of 
Education. This consortiu m also consists of the Teachers 
College at Columbia University, the University of Illinois, 
the University of Minnesota, RAND, Virginia Polytechnic 
Institute and State University, and the University of 
Wisconsin at Madison. In addition to its research and 
development agenda, NCRVE supports five dissem- 
ination and training programs: (1) dissemination; (2) 
professional development; (3) special populations; (4) 
planning, evaluation, and accountability; and (5) program 
development, curriculum, and instructional materials. 
This publication is part of our commitment to those on 
the front lines of vocational education, striving to create 
a new vision. 
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AUTHENTIC ASSESSMENT 
"This Is (Not Necessarily) a Test 



resting and Assessment in Secondary Education: 
A Critical Review of Emerging Practices by 
Clifford Hill and Eric Larsen begins its 
comprehensive investigation of authentic assessment by 
situating such assessment in terms of the purposes of 
testing. Ultimately, the authors have discovered, while 
tests and other forms of assessment differ in the type of 
information they seek and in the kinds of decisions they 
support, all tests inform decision making. 

In Testing and Assessment. Hill and Larsen describe one 
classroom-oriented purpose, or function, of testing — 
managing student learning — and two system-oriented 
functions — monitoring educational systems and 
evaluating students fo- itutional purposes. The 
practices represented by these functions, referred to as 
traditional testing, have played a role in education in the 
United States and throughout the world for the better 
part of this century. The multiple-choice test, referred to 
as conventional testing, has become tly k orototypical 
form of traditional testing. In recent years, however, 
educators have become aware of the reductionism 
perpetrated by conventional testing and have proposed 
alternative practices. Hill and Larsen refer to such 
alternative practices as authentic assessment, a term that 

conveys the notion that assessment should be 
built around tasks that are worth doing for their 
own sake. In determining intrinsic merit, 
educators derive standards from academic 
disciplines but. increasingly, from the larger 
society, as well. . . . 

Moreover, students are not expected simply to 
create discourse [verbal expressions, both 
written and oral]. . . . [Authentic assessment has 
developed tasks that go beyond the purely 
linguistic. Students are asked to make useful or 
pleasing art t fuels ... or to engage in 
performances. . . . 

In addition, Hill and Larsen believe 

it is useful to extend the concept of authenticity 
to include the perspective of students who are 
asked to take part in an assessment activity. . . 
Assessment should |also| be authentic . . . in the 
standards that are used in evaluating 
students. . . . |T|he standards ihat should be 



applied are those of practitioners ... in various 
academic disciplines . . . [and] in the workplace. 

In other words, fully authentic assessment has the 
following characteristics: 

• The tasks of authentic assessment themselves have 
intrinsic merit. 

• Students interact actively with the tasks prescribed 
by authentic assessment. 

• Practitioners would consider the standards applied 
to student interactions valid. 

With these three characteristics and their implications in 
mind, Testing and Assessment makes suggestions for 
improving the traditional scheme of testing and making 
it reflect authentic assessment. 

At the end of the first major section ol \ Testing and 
Assessment, Hi!! and Larsen define and challenge the 
testing paradigm itself, differentiating the subcategory 
testing from the broader category of assessment. As 
becomes clear in the explanation of the monograph's 
authentic assessment taxonomy (discussed below), 
testing does not equal assessment. 

Tests have four features: 

1 . Test givers conduct tests in a single time frame of 
specified duration. 

2. Tests consist of prescribed tasks that are presented 
in stable form. 

3. Tests elicit individual responses not based on 
external resources. 

4. E valuators evaluate test responses according to a 
pre-established scheme. 

The defining features of testing sometimes characterize 
examples of authentic assessment. However, Hill and 
Larsen go to great lengths to develop a taxonomy that 
makes a basic distinction between the practices of 
authentic assessment that remain within the testing 
paradigm and those that extend beyond it. 



NCRVE Change Agent 

10 



The authors devote the largest section of the monograph 
to a taxonomic overview of authentic assessment. In it, 
they present a set of heuristically oriented categories of 
authentic assessment in secondary education. Each 
mode of authentic assessment falls within one of two 
broad categories — alternative testing or documentation 
practices — established by the authors to differentiate 
between those that fall within the testing paradigm and 
those that "extend beyond'* testing, respectively. Table 1 
presents these categories hierarchically. 

The final section of Testing and Assessment evaluates 
authentic assessment in terms of three principles 
frequently used in discussing testing and assessment 
policy: excellence, equity, and efficiency. The authors 
also introduce a new term, "ecology." 

"Those uho support authentic assessment," Hill and 
Larsen contend, "place the pursuit of excellence at the 
heart of their policies." In terms of excellence and in 
contrast to conventional testing, authentic assessment 
requires students to construct responses rather than 
select among preexisting options, elicits higher-order 
thinking from students in addition to basic skills, uses 
direct assessment of holistic projects rather than indirect 



measures, and gets integrated with classroom instruction 
rather than separated from it. 

In terms of equity, educators who advocate authentic 
assessment claim that such assessment, which uses 
samples of student work collected over an extended 
period of time, defines clear criteria of which students 
are made aware, allows for the possibility of multiple 
human judgments, and more closely relates to what 
students learn in the classroom. 

Its advocates consider conventional testing highly 
efficient for two reasons: (1) the multiple choice format, 
which reduces the amount of time for and cost of 
administering and scoring tests, and (2) the statistical 
techniques known as psychometrics, which facilitate the 
construction of tests and the interpretation of results. 

Hill and Larsen, however, differ: 

From the standpoint of those who support 
authentic assessment, such notions of efficiency 
are superficial and misleading. It masks the 
massive inefficiency that arises in an educational 
system when fundamental goals are distorted by 



Table 1 . Taxonomy pf the Two "Branches" of Authentic Ass 



Alternative Testing 



Discourse 1 Testing 
Written Discourse 1 Oral Discourse 1 



Performance Testing 2 



With 

Intrinsic Discourse 3 



Without 
Intrinsic Discourse 3 



Documentation Practices 



Work Samples 
Portfolios Exhibitions 



Records 



System-Oriented 4 



Classroom-Oriented 5 



System- Classroom- Artifacts Performances Internships Competitions 
Oriented 4 Oriented 5 



' Discourse refers to verbal expressions, both written and oral (i.e., "questions" and "answers" in such tests are 
presented with words). 

1 Performance Testing elicits activity that goes beyond mere discourse. 

5 Intrinsic Discourse refers to discourse "built around tasks that are worth doing for their own sake." 

4 Syskm-Oriented refers to testing that results from needs outside of the classroom. 

5 Classroom-Oriented refers to testing that relates to classroum needs. 
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inappropriate methods of assessment. They thus 
argue for a deeper notion of efficiency, one that 
has to do with the degree to which assessment 
fosters good educational practices. 

Hence, the authors "prefer a deeper notion of efficiency 
that can be expressed by the term 'ecology'" — a term 
that they use not only in place of efficiency, but in place 
of excellence and equity as well: "It conveys an 
overarching principle that forces [educators] to examine 
the complex role that assessment plays within the larger 
educational enterprise " 

With this notion of ecology in mind. Hill and Larsen 
conclude their "assessment of authentic assessment" by 
focusing on the essential role authentic assessment plays 
in achieving three fundamental, or ecological, goah' of 
education: 

1. Reforming curriculum and instruction 

2. Improving teacher morale and performance 

3. Strengthening student commitment and capacity for 
self-monitoring 



The Testing and Assessment monograph ends with a 
caveat and a vision: 

Authentic assessment, when compared to 
conventional testing, makes far greater demands 
on both students and teachers. Indeed, those who 
support conventional testing have argued that 
authentic assessment consumes so much time 
that instruction is inevitably shortchanged. This 
way of thinking misconstrues the symbiotic 
relation between instruction and assessment. 
Each, when done properly, becomes the 
other: .... If this clinical spirit is to be 
humanely achieved in our classrooms, the very 
structure of education must be transformed. 
Such a transformation depends not only on our 
vision of what education should be but on the 
resources we are willing to commit to achieving 
this vision. 

Testing and Assessment in Secondary Education (MDS- 
237CA) was prepared by Clifford Hili and Eric Larsen from 
the NCR VE site at Teachers College, Columbia University. 
(For ordering information, see page 12.) 



COLLECTING ALTERNATIVE APPROACHES 
TO ASSESSMENT 



Traditional outcomes assessment in postsecondary 
vocational education has focused on outcomes 
that have routinely directed postsecondary 
vocational education programs — Job placement, 
occupational competence, program completion or 
retention, and earnings. As is the case in other sectors of 
the U.S. educational system, however, forces both inside 
and outside postsecondary vocational education 
advocate reform. Amidst the calls for reform by 
employers, a plethora of commissions, and other groups, 
the Carl D. Perkins Vocational and Applied Technology 
Education Act of 1990 mandates improved 
accountability systems, requiring states to measure 
student skills and performance in the areas of 
competency attainment, job or work skill attainment, 
retention or completion, or placement. 

In Alternate Approaches to Outcomes Assessment for 
Postsecondary Vocational Education* editor Debra 1). 
Bragg remarks that 



given the scope and intensit) of the educational 
reform movement, it is not surprising that 
institutions are being encouraged to look at a 
broader set of outcomes than had been used 
traditionally and to reexamine the means by 
which these outcomes were being measured. As 
new educational goals emerge to address reform 
directives and as new approaches to vocational 
education are employed, alternative approaches 
to outcomes assessment are essential. 

Alternative Approaches focuses on innovations in 
postsecondary vocational education outcomes 
assessment as identified during a iwo-year study at the 
NCRVE site at the University of Illinois. Bragg and her 
co-contributors intend their considerations primarily for 
local postsecondary education leaders — community 
college presidents, institutional researchers, and 
vocational education administrators — "who have the 
responsibility for conceptualizing and administering 
outcomes assessment, especially in vocational 
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education/' According to Bragg, "For this audience, 
[the] goal is to bring a new level of understanding about 
outcomes assessment and to encourage the exploration 
ot various approaches to measuring ouicomes Local, 
slate, and federal policymakers; government agency 
staff; program evaluators; researchers; teacher 
educators; and oi" .rrs involved in advocating, 
developing, or disseminating innovative methods of 
outcomes assessment for postsecondary vocational 
education would also find Alternative Approaches 
useful. 

The authors whose essays constitute the seven chapters 
oi Alternative Approaches write from the premise that 
there are three types of outcomes to consider when 
assessing postsecondary vocational education — 
institutional outcomes, program outcomes, and student 
outcomes. The chapters of Alternative Assessment 
address these three outcomes. As the abstracts that 
follow imply, each chapter focuses on an approach to 
outcomes assessment, suggests implementation issues 
and strategies, and discusses applications in one or more 
postsecondary settings. 

Ideas reported in Alternative App .niches, contends 
Bragg, 

represent what we believe are some of the 
newest and most promising ideas in how to 
conceptualize and conduct outcomes assessment 
in two-year postsecondary educational 
institutions. . . . Our primary purpose . . . was to 
describe various outcomes assessment 
approaches, to challenge you to explore them 
further, and to encourage you to consider how 
they can be applied to your institutions. 

A conclusion that summarizes the highlights of the 
chapters and offers some suggestions for implementing 
outcomes assessment in postsecondary vocational 
education follows the main chapters of Alternative 
Assessment. 

Chapter 1 

"Perspectives on Assessment Policy & Practice" 

Pebra D. Bragg and C. Michael Harmon 

To provide a basis for exploring alternative approaches 
in outcomes assessment. Chapter One discusses external 
forces on accountability, particularly those stemming 
from recent policy recommendations and legislative 
mandates. Then, internal forces, which largely reflect 
shifts in institutional missions and student populations, 
arc presented. With this background, research findings 



regarding outcomes assessment practices at selected 
two-year postsecondary institutions are presented. The 
chapter concludes with a discussion of challenges and 
recommendations for implementing outcomes 
assessment in postsecondary settings. 

Chapter 2 
"Total Quality Management" 

Debra D. Bragg 

Chapter Two advances the argument that the "Af " in 
TQM (total quality management) could just as easily 
stand for "measurement" as "management." This 
chapter discusses the philosophical and theoretical 
bases for TQM and describes how this comprehensive ■ 
approach can be implemented to enhance institutional 
effectiveness and ensure beneficial institutional 
outcome for students. Observations are made about 
how TQM is being implemented at Fox Valley Technical 
College in Wisconsin. 

Chapter 3 
"Assessing Student Success" 

C. Michael Harmon 

Chapter Three focuses on an institutional approach 
known as the student success model that parallels, at 
least philosophically, many of the tenets of TQM. While 
some readers may associate this model with a 
counseling and educational planning model, we believe 
that it can provide the basis for outcomes assessment as 
well. This chapter explores the way Santa Fe 
Community College (SFCC) in New Mexico has applied 
the student success model to develop and maintain 
institutional effectiveness and outcomes assessment. 

Chapter 4 
"Value-Added Assessment" 
C. Michael Harmon 

In Chapter Four, the concept of value added is applied 
to outcomes assessment and discussed in relation to 
institution, program, and student outcomes assessment. 
The value-added assessment approaches used by 
Delaware Technical Community College, Northeast 
Missouri State University, and Alvemo College are 
discussed. Issues surrounding the relationship between 
value-added and outcomes assessment are presented. In 
addition, the controversy about using pretest and 
posttest designs for measuring value added are explored 
and an argument for using performance assessment is 
made, . . , 
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Chapter 5 
"Concept Mapping" 

Thomas E. Grayson 

Chapter Five describe: a recent innovation that offers 
promise for getting program-level outcomes assessment 
efforts started, no matter whether they are based on a 
TQM, student success, value-added, or other approach. 
Referred to as concept mapping, this innovation is a 
bottom-up approach for producing a picture of how 
educational programs work and how they produce 
outcomes. This chapter illustrates how concept mapping 
was used to identify outcomes for vocational education 
at Black Hawk College in Illinois. 

Chapter 6 
"Outcomes Assessment 
in Vocational Education" 

N. L. McCaslin 

In Chapter Six, a schema for assessing educational, 
economic, and psycho-social outcomes for vocational 
education programs is presented. This chapter suggests 
that these outcomes must be considered in light of the 
educational needs and processes that influence 
vocational education. To conclude the chapter, a 



five-step process for conducting outcomes assessment is 
presented. 

Chapter 7 
"Performance Assessment" 
Linda Mabry 

Chapter Seven addresses the contemporary topic of 
performance of student knowledge and skills. This 
chapter defines common terms such as ''performance 
assessment, " "alternative assessment, " and "authentic 
assessment. " The chapter describes assessment 
techniques such as profiles and records of achievement, 
portfolios, and performance tasks. The chapter also 
discusses how these assessment techniques are being 
implemented in several secondary and post secondary 
organizations and describes issues surrounding their 
use in these settings. 

Alternative Approaches to Outcomes Assessment for 
Postsecondary Vocational Education (MDS-239CA) was 
edited by Debn D. Bragg from the NCRVE site at the 
University of Illinois at Urbana-Champaign, (For ordering 
information, see pace 12j 



"LET ME GET THIS STRAIGHT" 

What Perkins Accountability Mandates Really Mean 



EJfandora s Box: Accountability and Performance 
Standards in Vocational Education by Paul T. 
Hill, James Harvey, and Amy Praskac presents 
the results of a study on accountability in vocational 
education. RAND, a nonproft research institution 
established to "further and promote scientific, 
educational, and charitable purposes . . . for the public 
welfare," and an active participant in the NCRVE 
consortium, conducted the study on behalf of NCRVE. 

The Pandora s Box study "focused on the to who.n' 
side of accountability, trying to determine who needed 
evidence about the performance of vocational education 
and how they would use that evidence/' The study had 
two goals: ( 1 ) "to understand the intentions of the 
people who formulated and enacted the requirements for 
vocational education performance standards and 
accountability" contained in Perkins legislation and i2) 
"to understand how those requirements would affect 
state and local accountability processes in vocational 
education." What began, however, as 



an examination of the potential uses for 
vocational education performance information 
by [federal] and state officials, local educators, 
employers, parents, and students . . . turned 
quickly ... to an examination of two prior 
I issues:] 

1 . the ways in which states and localities deal with 
changes in state administration — as required by the 
1990 Perkins Amendments — and the extent to 
which they can advance accountability through 
performance standards 

2. the questions "raised repeatedly" during interviews 
for the Pandora s Box study about whether the 
Perkins Amendments "imply the need to develop 
statewide aggregate measures of program 
performance and, ultimately, national aggregate 
measures of performance a^ the foundation of 
educational accountability" 
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NCRVE considers Pandora \y Box a companion to other 
NCRVE projects and publications that have examined 
the "for what** of vocational education accountability. 
These "companions" include Performance Measures 

Information about the 
needs of broader regional, 
statewide, or national 
labor markets can help 
educators and students 
understand opportunities 
that might not be 
evident locally 



and Standards for Vocational Education as well as 
Local Accountability in Vocational Education: A 
Theoretical Model and Its Limitations in Practice and 
Beyond Vocational Education Standards and Measures: 
Strengthening Local Accountability Systems for 
Prog.^m Improvement, the latter two of which are 
reviewed together in this issue of Change Agent (for that 
review, see pages 9-12; for ordering information about 
all three publications, see page 12). As its major 
conclusion. Pandora 's Box found that 

tb- main focus of accountability in vocational 
education is local, and that the Perkins 
(legislation! requirements for new performance 
measures and standards are meant to strengthen 
local accountability, not to weal, it or to create 
new channels of reporting to the [federal) 
government. 

Purpose of Perkins Accountability Requirements 

Hill, Har\ev. and Pra^kac explored the purpose of 
Perkins accountability mandates using interviews with 
Congressional staff members and lobbyists. The 
interviews made it clear that no one intended to 
construct a stream of information to Washington. DC. 
that would let Congress or anyone else manage 
vocational education from afar. The drafters of the act 
also did not intend to obtain a master database from 
which they could derive an assessment of vocational 
education nationwide or in an\ particulai locality or 
state. 



It became clear, however, that Congressional members 
and senior committee staff had concerns about the 
quality of local programs and about whether they were 
being implemented with sufficient concern for the needs 
of local students and employers. Interviewees based 
their concerns on business leaders' complaints about the 
c, ality of high school graduates (including graduates of 
vocational programs), and on interviewees' own fears 
that some local managers of vocational education had 
become "bureaucratic, deadened by routine, and more 
concerned with protecting staff jobs than with meeting 
local labor market needs.'* 

Through the Perkins requirements, legislative supporters 
of vocational education intended to reassure critics that 
Congress considered the quality of local programs 
important and "lax or self-indulgent" program 
administration intolerable. At the same time, however, 
members of Congress did not want to establish 
themselves as "the remote, ultimate judges of local 
processes." Instead, they wanted to create a framework 
that would guarantee attention to quality issues at the 
local and state levels. 

Effects of Perkins on Local and State Accountability 

Results of case studies in seven states, including both 
urban and rural localities, showed that, local 
accountability processes vary in quality. Some localities 

Perkins requirements for 

new performance 
measures and standards 
are meant to strengthen 
local accountability, not to 
weaken it 

ensured the quality and relevance of vocational 
education programs through processes such as the 
following: 

Employer advisory committees 

Tracking of employer needs through student 
placement 

0 

Attracting students via job placement success 
Employer certification of the need for new courses 
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Eliminating or upgrading courses with low 
enrollment or poor placement records 

However, many local accountability processes have 
serious weaknesses, including a "routine-following" 

Accountability 
in vocational education 
focuses mainly on the 
local level 

mindset and lax tracking of student placement among 
vocational education providers. 

Also, real conflicts of interest exist among local 
constituencies. In very small communities with only a 
few employers, healthy accountability processes often 
work in everyone's interest. In larger communities, 
however, only a fraction of employers can be consulted 
directly. As a result, important employer interests 
remain poorly represented. 

Conflicts of interest among students and employers also 
arise. Established employers sometimes want students 
prepared for traditional low- and moderate-skilled jobs, 
preparation that might prevent . .udents from obtaining 
jobs with employers who require nontraditional or 
advanced skills, possibly in other localities. 

"These conflicts of interest," contend the auihors of 
Pandora *s Box. 

justify oversight by state and federal 
governments. The state needs to protect its 
broader economic future against the narrow 
localism and short-time horizon that can 
dominate local accountability processes. The 
state and federal governments need to help 
disadvantaged groups that lack the leverage to 
protect themselves at the local level. 

State and federal governments can also help to expand 
the horizons of local actors in several ways. Information 
about future economic trends can help employers to 
anticipate their own needs. Information about the needs 
of broader regional, statewide, or national labor markets 
also can help educators and students understand 
opportunities that might not be evident locally. Both 
lists of competencies required by employers and testing 
systems that assess performance in light of those needs 
can create demands of program imprmcmcnt. 



Competency-based systems can both raise standards and 
provide concrete guides for program improvement. 
Program developers derive such systems from lists of 
competencies created by knowledgeable employers. 
When used as the basis for curriculum development, 
student testing, and certification, these lists guide 
vocational education policy and build employers* 
competence. They also provide external standards for 
evaluating program quality and student access. 

Based on their case studies, however. Hill, Harvey, and 
Praskac concluded that "hard outcomes data should 
supplement, not supphnt, processes of consultation and 
monitoring and the use of managerial judgment." 

In the last chapter of the monograph. Hill, Harvey, and 
Praskac present the conclusions of and draw 
implications from their research. The topics of 
conclusions and implications concern national data 
systems, political accountability and program 
improvement, aggregating information, state and federal 
assistance, the relation of outcomes data and local 
processes, and monitored versus "automated" 
accountability. 

All in all. Pandora's Box should go quite a way in 
alleviating the fears of many educators in regards to the 

Congress's concern about 
the quality of local 
programs was based on 
business leaders' 
complaints about the 
quality of high-school 
graduates 

Perkins requirements while contributing to important 
discussions about accountability in vocational 
education. 

Pandora's Pox: .Arc onntahdit\ and Performance Standards 
in Vocational Education (MDS-2SSCA) vkjs prepared b;> 
Paul T. Hill. James Har\e>. and Ann Praskac Irom the 
NCRYH site at RAND. ( For ordering informal ion. see page 
I2.i 




NCRVE Change Agent 



ACCOUNTABILITY 
Local Perspectives 



How can we best foster accountability in 
vocational education? What is the 
nature of existing accountability 
relationships? Concern about the productivity of 
the U.S. workforce and the competitiveness of 
the U.S. economy heightens the importance of 
these questions. Recent reports from the U.S. 
Department of Labor and the National 
Educational Goals Panel focused public 
attention on the performance of the vocational 
training system. Legislators at state and federal 
levels are formulating policies aimed at 
improving vocational education through 
increased accountability. The policies will be 
better informed if they are based on thorough 
knowledge of existing accountability systems. 

So begins the first of two NCRVE reports on 
accountability, both by Brian M. Stecher and Lawrence 
M. Hanser of RAND. 

As its title implies, the first report. Local Accountability 
in Vocational Education: A Theoretical Model and Its 
Limitations in Practice, describes a model for local 
accountability in vocational education and explores 
shortcomings of accountability systems in general, The 
companion report. Beyond Vocational Education 
Standards and Measures: Strengthening Local 
Accountability Systems for Program Improvements 
considers the demands that federal legislation places on 
localities and states and proposes policy alternatives for 
responding to those demands that are influenced by 
local accountability concerns 

Local Accountability in Vocational Education 

Intended tor local and state administrators, state and 
federal policymakers, and vol itional education policv 
researchers, the Local Accountability study analyzes and 
describes local systems of accountability in vocational 
education. "We know very little about accountability in 
vocational programs Stecher and Hanser assert, 
''There is no clear picture of the functioning of . . . local 
accountability systems." Believing as the\ claim, that 
"policies will be better informed if they are based on 
thorough knowledge of existing accountability 
s\ stems." the authors contend that the program 
descriptions of Local Accountability "ser\e as the basis 
lor further study of vocational programs, lor developing 



criteria for evaluating accountability systems, and for 
monitoring the impact of policy changes." 

Most formal policy initiatives in educational 
accountability focus on "highly aggregated" 
accountability at the state or federal level. Legislation 
and other initiatives can also, however, define 
accountability systems in local terms. The Carl D. 
Perkins Vocational and Applied Technology Education 
Act of 1990 promotes both local and state 
accountability, placing with states "the final 
responsibility for establishing measures and standards 
and for ultimate program supervision after placing 
initial responsibility at the local level for program 
evaluation and improvement." 

Because of their importance in understanding the impact 
of this model of accountability, the manner in which 
vocational programs presently function with respect to 

We can identify many 
practical constraints that 
reduce the effectiveness of 
the components in a local 

accountability system 

their multiple constituencies require examination. Local 
Accountability analyzes the extent to which local 
accountability systems exist in vocational education and 
describes the nature of the underlying relationships 
between such programs and their constituents. In 
exploring local accountability, Stecher and Hanser 
visited secondary and postsecondary vocational 
programs in California, Florida, Michigan, Ohio, and 
Oklahoma. They gathered information about programs 
and interviewed students, parents, instructors, 
employers, and administrators. They also interviewed 
staff at the departments of education for each state. 
Interviews explored the relationships between the 
programs and local constituencies. 

four major conclusions emerged from the Local 
Accountability study, first, the stud) found widespread 
evidence of functioning accountability systems in 
\oca t: onal programs at the local level. Second, foi r 
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elements — goals, measures, information feedback loops, 
and change mechanisms — and the relationships among 
them describe these accountability systems. Stecher and 
Hanser have developed a simple model based on these 
elements that does a reasonable job of describing 
accountability relationships across a spectrum of 
vocational programs. Third, the quality of these 
elements and the relationships among them account for 
much of the variation in local accountability systems. 
Limitations in these components, the authors found. 

Policies will be better 
informed if they are based 
on thorough knowledge 
of accountability systems 
as they currently exist 

interfere with the overall effectiveness of the 
accountability system. Fourth, the authors identify many 
practical constraints that reduce the effectiveness of the 
elements in local systems of accountability. 
Identification of these elements provides a basis for 
developing both criteria for evaluating local 
accountability systems and prescriptions for improving 
them. 

Stecher and Hanser found that programs had 
deficiencies in each of the elements of the model the 
authors developed and in the interactions among those 
elements. Specifically, the study found goals ineffective 
when the following occurred: 

interlocking goals at the action (local) level did not 

• support higher-level goals 

the broadness or vagueness of the goals made it 
0 impossible to know when or if they had been 
achieved 

relevant constituencies did not understand and/or 

• support the goals 

the priorities among the goals remained unstated or 

• unclear 

In their chief role, measures provide evidence of j-oal 
attainment; the most important measures relate directly 
to established goals. The study found measures 
ineffective when they lacked the following: 



consonance with or did not sufficiently encompass 
° goals 

adequate technical quality 

meaning for constituents 

Feedback conveys information about measures to 
administrators, program staff, and school system 
constituents as well as among administrators and staff. 
Potential deficiencies in feedback include the following: 

insufficient communication 

inaccurate communication 

low signal-to-noise ratio — a high rate of 

• communication with very little useful content 

Ultimately, accountability systems lead to reform in 
organizations. Organizational change mechanisms can 
fa'ter as a result of the following: 

regulations that limit options for change 

e 

insufficient resources 

overattention to the needs of one constituent" (e.g., 

• employer groups) 

priority to short-term demand rather than long-term 

• trends 

difficulties in balancing competing goals and 

• principles (e.g., equity versus placement) 

lack of formal procedures for change 

0 

ineffective leadership for reform 

* 

"Greater experience with this model of local 
accountability will improve its usefulness as an analytic 
tool." contend Stecher and Hanser. For example, the 
model can serve as a basis for monitoring the effects of 
state and federal initiatives." including the l c )9() Perkins 
Amendments. 

The follow-up companion to Local Accountability, 
Beyond Vocational Education Standards and Measure*. 
does in fact work from this model of local 
accountability in monitoring such effects. 

While the report's summary provides a useful outline of 
Local Accountability'** main points, a more thorough. 
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specific, and valuable discussion constitutes the bulk of 
the report and warrants a thorough examination. 

Beyond Vocational Education 
Standards and Measures 

Beyond Vocational Education Standards and Measures: 
Strengthening Local Accountability Systems for 
Program Improvement, also by Stecher and Hanser, 
examines vocational education accountability at the 
local level in an effort to provide information that might 
improve the implementation of Perkins 1 Amendments 
provisions. 

Results of the study show that local accountability 
systems exist in many vocational programs. While 
existing local accountability systems do evince value in 

Most vocational programs 
have appropriate local 
accountability systems 

improving programs and can, in fact, accommodate the 
mandates of the Perkins Amendments, such systems are 
less accommodating of other, often broader Perkins 
mandates. 

Written for local practitioners involved in adapting the 
new Perkins accountability requirements, state 
policymakers charged with implementing them, and 
federal policymakers involved in 1995 reauthorization 
of vocational legislation, the authors of Beyond . . . 
Standards suggest ways that local and state programs 
"can improve the functioning of local accountability 
systems" and recommend "changes in federal vocational 
education policy to further the goals of program 
improvement" that motivated the 1990 Perkins 
Amendments 

"I'nfortunateK." Steelier and Hanser observe. 

little is known about the local program context 
and the likely interplay between local conditions 
and Perkins [legislation] accountability 
mandates, '''he sludy reported here explores 
accountability from the local perspective to 
pro\ide insights into the ways in which local 
programs define their goals, interact with thetr 
constituents, use information, and make 
pror ram -related decisions. It also attempts to 
define the role stales could pla\ in support ot 
these functions. 



[We] developed a model of the local 
accountability systems currently operating in 
vocational education programs. The model 
illustrates how goals, measures, f- edback, 
influence, and reform interact to influence 
program practice. The model also offers 
guidance for evaluation and strengthening local 
accountability systems. Finally, it can be used as 
a basis for analyzing the interplay between 
existing global accountability and accountability 
as envisioned under [the Perkins Amendments]. 

Stecher and Hanser draw five conclusions based on their 
analysis of federal policies, local accountability systems, 
and the interactions between them; the conclusions are 
addressed in detail in the Beyond . . . Standards report: 

1 . Most vocational programs have functioning local 
accountability systems consistent with the model 
envisioned in the Perkins Amendments. 

2. Local accountability systems can be effective tools 
for program improvement if practical obstacles can 
be overcome. 

3. States can do much to evaluate and strengthen local 
accountability systems. 

4. Neither state education agencies (SEAs) nor local 
education agencies (LEAs) are fully prepared to 
carry out their responsibilities vis-d-vis 
accountability under the Perkins Amendments. 

5. The Perkins Amendments may overemphasize 
state-adopted outcome-based standards and 
measures. 

Beyond . . . Standards concludes with an impressive set 
of implications and with recommendations about 
modifying Perkins and associ jd legislation. 

lx>cal Accountability in Vocational Education: A Theoretical 
Model and Its Implications (MDS-291CA) and Beyond 
Vocational Education Standards and Measures; 
Strengthening lx>cal Accountability Systems for Program 
Improvement (MDS-292CA) were prepared by Brian M. 
Stecher and Lawrence M. Hanser from the NCRVK site at 
RAMI), (lor ordering information, seepage 12.) 
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INTRODUCING A NEW NCRVE MINICATALOG 



Earlier this year, President Clinton signed into law an 
important piece of education legislation, the School-to- 
Work Opportunities Act. While the National Center for 
Research in Vocational Education (NCRVE) has 
continually sponsored projects dealing with school-to- 
work issues and initiatives, only recently has NCRVE 
begun to distribute a school-to-work minicatalog. 

Like NCRVE's other minicatalogs, the "School-to- 
Work" catalog highlights an important area of 
vocational education in which NCRVE has produced a 
significant amount of conceptual and practical thinking. 
NCRVE has organized "School-to-Work" around the 



central themes of the School-to-Work Opportunities 
Act — school-based learning, work-based learning, and 
connecting activities — as well as the issues of 
accountability and assessment. 

This issue of Change Agent reviews six recent 
NCRVE publications included in the "School-to-Work" 
catalog. The free "School-to- Work" catalog is available 
from the NCRVE Materials Distribution Service (see 
page 12 for details). 



AN ANNUAL REPORT 

ON SCHOOl-TO-WORK PROGRAMS 

School-Based Learning 



The 1990 Amendments to the Carl D. Perkins 
Vocational and Applied Technology Act require 
the National Center for Research in Vocational 
Education (NCRVE) to prepare an annual report of the 
research conducted on successful education -to- work 
transition approaches. Research on School -to-Work 
Transition Programs in the United States by David 
Stern. Neal Einkelstein. James R. Stone 111. John 
Latting, and Carolyn Dornsife is the 1994 fulfillment oi 
tnul requirement — it is also an example of the NCRVE 
publications listed in the "School-Based Learning" 
section of NCRVE' s new "School -to- Work" 
minicatalog. 

Research on School-to-Work opens with a briel 
overview of "the difficulties encounteied hy many 
young people in making the transition from school to 
work." Such difficulties include or are exacerbated by 
an increasing scarcity of career jobs, reduced 
employment in the manufacturing sector, employers' 
increased demands for cognitive ability on the part of 
employees, and the growing disparity between the 
earnings of average high school graduates and those of 
high school graduates w ho are more competent in 
mathematics or who lio on to ultimo from foui-year 
colleges. 

The most immediate implication ol tins situation, 
according to Research on Sclutol-to-W ork. is that the 



success of school-to-work programs cannot be judged 
simply in terms of students' first full-time jobs; instead, 
the success of school -to- work programs needs to be 
gauged in terms lifetime employment histories — by 
how these programs prepare students for lifelong 
learning and how the programs propurc students with the 
knowledge and skills necessary to change jobs. 

Research on School-to-Work adopts a narrower 
definition of school-to-work than one that would include 
"any and all efforts that contribute to successful school- 
to-work transition." A discussion based on such a 
delinilion would be unw ieldy since the definition 
"w ould encompass all kinds of schooling and non- 
school education that help in any way to prepare for 
working life." Therefore. Research on School-to-Work 
focuses on the second of two main categories oi' school- 
to-work programs — the first of which, school-urn/- work 
programs, simply "allow students to work and attend 
school during the same lime period." 

Research on Silu>ol-to-Work instead equates school-to- 
work with the second category, school- jor* work 
programs "education and training programs in which 
preparation for work is explicitly a major purpose." 
More narrowly, the report focuses on work preparation 
programs that sorxc high school students, 
nonhaccalaiircale posisocondary students, or out-ol 
school youth. Specifically . it locuscs on cooperative 
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education, >outh apprenticeship, other work experience, 
school-based enterprise, Tech Prep, career academies, 
and school-to-apprenticeship. 

A key element of most school-to-work programs is the 
combination of school and work in some form during 
the same period of time. The following three 
programmatic features are especially important in 
determining the nature of the programs and the kinds of 
students who participate: 

1 . integration of school-based and work-based 
learning 

2. combined vocational and academic curriculum 

3. linking of secondary and postsecondary education 

Each of these features is discussed in detail in the first 
section of the report. 

Discussion of the various program types addressed by 
Research on School -to-Work is structured within the 
framework of four "programs*' sections: The first, 
"School-and-Work Programs in Secondary Schools," 
covers cooperative education, youth apprenticeship, 
school-based enterprise, and non-school-superviscd 
work experience. In addition, the section discusses 
issues lelating to career counseling, job placement, and 
mentoring programs as well as the results of an NCRVE 
longitudinal survey that studied the effects of students 
working while in high school. 

The second of the four sections, "SehooI-and-Work 
Programs in Two Year Colleges," covers cooperative 
education programs and apprenticeship programs as 
well as career counseling and job placement in two-year 
colleges, all of w hich, the report posits, 

are taking increasingly diverse roles in the 
delivery of job-related instruction. While it has 
always been the case that comnuinit), junior, 
and technical colleges offered some vocational 
curricula, the range of direct linkages with 
outside organizations [including employment 
providers) has become remarkably wide. 

The third section, "Relevance of Vocational Education 
to Subsequent Employment" explains how- 
cooperative education, apprenticeship, or other 
school-and-work progiams engage students in 
school and work during the same period of time. 
The relationship between school and work is 



concurrent. In addition, high schools and two- 
year colleges, particularly their vocational 
education programs, include preparation for 
work as a major part of their mission. They are 
"chool-for-work programs. If they succeed, there 
should be a sequential connection between 
school and work. Evidence about this sequential 
link is reviewed in this section. 

In the fourth section, "Programs for Out-of-School 
Youth," Research on School-lo-Work briefly explores 
the numerous programs that "have attempted to improve 
the job prospects of young people who are not in school 
and are looking for work." Based on a study of relevant 
literature, ihe report concludes that while some 
programs have a record of improving earnings, "the 
resources and structures of these programs are largely 
unable to overcome problems related to family structure, 
neighborhood structure, education, and health. 1 * 

For each of the program types discussed — cooperative 
education, youth apprenticeship, other work experience, 
school-based enterprise, Tech Prep, career academies, 
and school-to-apprenticeship — Research on School-lo- 
Work provides 

The success of school-to- 
work programs cannot be 
judged simply in terms of 
students' first full-time 
jobs; instead, success of 
such programs needs to 
be gauged in terms of 
lifetime employment 
histories 



a definition and history; summarizes important related 
research findings; assesses the value of the program type 
as an educational initiative; highlights what the report 
calls "elements of good practice," describes examples of 
program implementation in the United States, and/or 
discusses emerging issues. 
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Choosing a School-to-VVork System 

The conclusion of Research on School -to-Work poses an 
important question about w hat local and state 
decisionmakers can learn from the research that may 
inform the strategic decisions they face in designing 
new school-to- work systems. To this, the authors of the 
report answer. 

Probably the most fundamental decision is 
whether the new system will be designed only 
for the "non-college-bound" or whether it will 
be more inclusive. This choice has a direct 
bearing on what kind of school-based instruction 
and what kind of work-based learning should he 
offered. 

These issues are the foundation of the major findings of 
Research on SchooI-lo-Work Transition Programs in 



While it has always been 
the case that community, 
junior, and technical 
colleges offered some 
vocational curricula, the 

range of linkages with 
outside organizations has 
become remarkably wide 



ease of access to and flexibility in transitioning 
from a school -to- work system 

• the nature of the school-to-work curriculum, 
including where and how it will be delivered 

• the relative emphasis on learning to do work versus 
learning to think 

Ultimately, the report concludes. 

the questions are complex, and there are few 
definitive answers. Even w hen programs have 
been found to be effective, it is usually not 
possible to say exacth w hy they are effective 
because the programs themselves are 
multifaceted [and] each program model . . . has 
been implemented differently in different 
places. . . . If new school-to-work programs can 
improve students' performance, public 
authorities need not worr> about whether certain 
program components are included. 

Finally, it is useful to recall that school-io-work 
transition seldom means an abrupt transition 
from full-time schooling to full-time 
employment. The initial transition from school 
to work usually occurs over a period of several 
years, during which work is combined with 
school. A successful school-to-work transition 
s\ stein will use this initial transition period to 
help young people find and keep the kind of full- 
time job thes want, with a minimum of wasted 
time. But it will also do moie than that. A 
successful system will enable >oung people to 
master the process of learning while they work. 
In a fasl-chaneinu economs. this is fundamental. 



the United States. Among the issues to be considered in 
the strategic choices that decionsmakers will ha\e to 
make are the following: 

the role of vocational education in general in the 
United States 

• the nature of the students w horn school-to- w ork 
programs are to prepare and those students' 
relationship to students whom school-to-work 
programs will not ser\e 

• the image of a new. supposed!) comprehensive 
school-to- work system in the e\es of students, 
educators, and the public in general 



Research on Sch<}o!~t<t~\\'ork Transititm rroxrams in the 
t'nited Stun • (MDS-771CA) was prepared b\ Da\id 
Stern. Neal i inkelstein. James R. Stone III. John Latting. 
and Carolyn Dornsife l v om the NCR VK site at the 
l'ni\ersit\ of California at Berkeley (l ; or ordeimg 
information, see page 12.) 
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DIVE RIGHT IN! 
Work-Based Learning 



Vocational education and training must 
prepare workers to function effectively 
in . . . new environments; reciprocally. 
. , , new work environments must draw on. 
reinforce, and encourage the further 
development of the skills, knowledge, and 
behaviors addressed r>\ education and training 
programs. 

This is one contention of a recent NCRVE publication 
that explores another area of school-to-work transition 
— work-based learning. Learning the Ropes: The Social 
Construction of Work-Based Learning by Sylvia Hart- 
Landsberg. Jane Bra unger. Stephen Reder, and Mary M. 
Cross argues the importance of work-based learning, 
raises some key issues in this area, and offers a 
framework for assessment, finally putting that 
framework to the test with medical unit secretaries in a 
large private hospital — an occupational group and work 
setting "characterized by considerable learning the 
ropes." 

The significance of and need for work-based learning. 
Learning the Ropes contends, can be understood in both 
practical and theoretical terms. Practically speaking, 
much of the short-term future workforce is already on 
the job; demographic projections about the future 
workforce indicate a substantial need for worker 
training and education in the short-term future; and 
according to projections, increasing percentages of 

Learning the Ropes is as 
interesting for its activity 
theory framework as it is 
for its conclusions 

entering workers will lack the basic knowledge and 
skills that high-wage jobs require. ' Ultimately, if 
workers and companies are to remain competitive. . . . 
significant investments in work-based education and 
training will likely be required." On the theoretical side, 
successful efforts to coordinate educational and 
workplace restructuring will likely require innovative 



and far-ranging but specific theories about the 
relationships between education, training, and work and 
about the processes that occur with each. 

With the importance of work-based learning established. 
Learning the Ropes briefly outlines four key issues in 
the area of work-based learning: 

1. distinguishing learning from doing in workplace 
settings 

2. specifying de'- elopmental relationships between 
effective and cognitive components of work-based 
learning 

3. identifying the social organization of expertise in 
workplaces 

4. understanding the relationship between individual 
learning and organizational development 

In the ethnographic research done for the Learning the 
Ropes, each of these issues is addressed in detail. 

The framework employed by the Learning the Ropes 
researchers is based in "activity theory." which takes 
human activity as the "basic unit of analysis" in efforts 
to understand behavior and the mind. In activity theory 
approaches, an activity system — which is constituted by 
a subject (or practitioner), an object (motive), mediating 
artifacts (tool, signs, and symbols), rules, a community, 
and a division of labor — is analyzed on both individual 
and community levels. Learning the Ropes is as 
interesting for this framework as it "is for its conclusions. 

The oft-used informal term "learning the ropes" 
suggests one of many important types of workplace 
learning processes. Learning the ropes is unique as a 
workplace learning process in that it is accomplished in 
a largely "unsupported" manner, "without benefit of 
either the master-learner format of apprenticeship 
learning or the guided practice characteristic of on the 
job training." Individuals learn the ropes by "figuring 
out how things work in an organization, learning to 
whom one goes for help in dealing with various kinds of 
problems . . ., learning how lo collaborate (in an] 
'organizational culture/" and so forth. This, then, is the 
paradigm by which the work-based learning of a group 
of novice and experienced medical unit secretaries was 
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studied — through observation, interviews, and 
document collection. 

The conclusions of Learning the Ropes fall into ihree 
categories: (I) the connections between learning and 
doing, (2) the role of the activity system in the learning 
process, and (3) the impact on the learner and the 
activity system of unsupported learning experiences. 
These findings indicate that learning the ropes, much 
more than an accretion of skills or procedural 
knowledge, is a matter <. f coming to understand the 
work and its context as a . . . developing system." 

1. Learning and doing 

0 Learning the job is a major part of the job 

° Learning and doing are concurrent: an 

antecedent/consequent model of training and 
work is not applicable to the hospital units 
studied 

Learning the ropes entails developing four 
capabilities that underlie the required skills for 
unit secretaries (organi/.ing the workload, 
relating to people, learning the organization of 
the unit and hospital, and developing a concept 
of self in relation to work and the organization) 

2. Refleclion-in-aclion in the activity s\sleui 

• A large part of learning the ropes is learning the 
activity system 

• Learning in the activity system occurs through 
a process of reflection -in-aclion that is largely 
self-initialed 

The nature and structure 
of the knowledge that 
accrues with experience 
requires clarification 

• A unit secretary who is learning the ropes 
participates in the ongoing interpretation and 
negotiation of the acti\it\ sWem 

Learning about each component of the acti\it\ 
system is a driving force m the activity sWeni 



3. Supported and unsupported learning in the activity 
system 

• Learners see themselves as constantly engaged 
in learning, but as seldom being taught; 
however, in this community, each individual is 
a teacher and each is a learner 

• Unsupported learning is the norm here, even 
though there is much assisted performance 

Keeping in mind the conclusion of Learning the 
Ropes — that learning the ropes is a process of coming to 
understand the "big picture" of a workplace s> stern 
rather than simply "the accretion of skills or procedural 
knowledge" — school-to-workplace training can extend 
the learning that aheady takes place by 

• recognizing the essential role of assisted learning 

• creating conditions to foster assisted learning 

• designing and implementing innovations that 
structure learning in the workplace 

° designing and implementing innovations thai 
structure occupational learning in schools 

Like the conclusions that lead to them, these 
implications are discussed in detail in the report. 

The authors of Learning the Ropes situate their work as 
the first of what will necessariU he many investigations 
of learning the ropes as a l\pe of work-based learning. 
Stud)ing "both the nature and the content of 'the ropes' 
that experienced workers have learned as well as a 
means through which new workers learn those 'ropes' 
over time" is required. The nature and structure of the 
knowledge that accrues with experience will require 
clarification. Ultimately, the manner in which new 
workers acquire "ropes" knowledge and the manner in 
w hich literacy, communication, and job-related skills 
impact that acquisition over time will require 
longitudinal study. 

Learning the Rof>e\: The Sot ml ('tmsttm turn <>l Woik 
Based Learning (MDS-41 was prepared h\ S\l\ ki 
Hail -Laiulshcrg. Jane Uraiinger, aiul Stephen Kcdci of the 
Northwest Regional Ldiicalton l.,thoinlor\ ami M.u \ M. 
Cross from the NCKVL site at the I ni\ei sit\ ul 
California at Beikekw ( l or ordering information, see 
pare 12.i 
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SPECIAL POPULATIONS AND MORE 
Connecting Activities 



There are a number of issues and concerns not direct !\ 
or exclusive!} related to school-to-work initiatives that 
are nonetheless important to the success of school-to- 
work programs. Decisionmakers and practitioners will 
find NCRVE publications concerning such issues useful 
and even necessary ;is the\ work to develop a successful 
national school-to-work system and implement it at the 
local and state le\el. Three of these publications — 
Gender Equity: A Resource List of Organizations and 
Information Centers and the two volumes of Exemplary 
Programs Serving Special Populations — address two 
such issues. 

Hauler Equity, compiled b\ the NCRVE Office of 
Special Populations (OSP) at the University of Illinois, 
is one in a series of ongoing efforts of this office 
designed to contribute to four of the OSP* s objectives: 

1. To increase awareness ami understanding ol critical 
issues in vocational special needs education 

2. To increase the use of available resources 

.V To initiate support networks of professionals 

4. To promote exemplarv program acliv its and the 
adoption of model practices 

In their introduction. Gender Equity* s compilers outline 
the significance of gender issues in vocational 
education: 

(lender equitv remains a challenge to educators, 
employers, administrators, and ser\ ice piovidcrs 
as thes respond to the growing diversity in 
today's workforce, changes in society, and 
economic demands. It means the existence of 
conditions that gi\e girls and boys, women ami 
men the same opportunities and choices to 
advance themselves in education, training and 
careers . . based on their abilities and talents 
and not on gender role stereotypes and 
expectations 

By the turn ol ilk ccntuiv. women aic expected 
to account lor nearly hall ol the labor lorce. The 
LouMiiinenl is moving to meet the giowing 
need* ol this population by enacting the (ail 1). 
Peikins Vocational and Applied I ethnology 



Education Act of 1990 and introducing the 
Gender Education Equity Act (H.R. 1793) of 
1993 in Congress. The 1990 Perkins Act 
contains provisions designed to increase the 
education and career opportunities of girls and 
women in vocational education. H.R. 1793 
addresses the issues of gender bias and inequity 
w it ti in the educational system that negatively 
affects girls ami women. It contains nine bills, 
including the Women's Education Equity Act 
(WEEA). In addition to such positive legislative 
initiatives, a number of organizations, 
institutions, educational information/research 
centers, and special interest groups have been 
strong advocates for the rights of girls and 
women. 

While intended to be selective rather than 
comprehensive, tlu guide itself contains descriptions 
and contact information for over 160 organizations, 
educational information centers, and human resources 
focusing on gender equity. 

^ ^ 

Each of the two volumes of Exemplary Programs 
Serving Spa-ad Populations (Volume I by Zipura T. 
Burac and Volume 2 by Zipura T. Burac and Robert 
Yanello, both from OSP at the University of Illinois) 
highlights five exceptional programs serving individuals 
from special populations selected by OSP in 1990 and 
Wl. OSP searches for and provides information about 
exemplary programs as part of its mission of improving 
vocational programs. 

Designed for administrators, state personnel, program 
coordinators, practitioners, and researchers, the two 
volumes were developed to 

• disseminate information about exemplary programs 

• prov ide models that can be adapted by those 
interested m dev eloping programs 

• prov ide specific examples of innovative practices 
and strategies that other programs can emulate 

In addition to the information about the programs 
themselves. Exemplary Programs also provides 
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background on the search for the programs, including a 
discussion of the framework by which the programs 
were identified. 

Gender Equity and the Exemplary Programs volumes 
are just three examples of the many NCRVE 
publications that are contributing and will continue to 
contribute to the development of school-to-work 
programs in the United States. Below is a categorized 
list of all publications listed in the "Special Populations" 
division of the "Connecting Activities" section of 
NCRVE's "School-to-Work" minicatalog. They are all 
available from the NCRVE Materials Distribution 
Service (for contact information, see page 12). 

Special Populations — General 

Increasing Vocational Options for Students with 

Learning Handicaps: A Practiced Guide 
Resources To Facilitate the Transition of Learners with 

Special Needs from School-to-Work or 

Post secondary Education 
Selected Resources To Facilitate the Transition of 

Learners with Special Needs from School-to-Work 

or Post secondary Education, VoL 2 

Special Populations — Corrections 

Bright Hopes, Dim Realities; Vocational Innovation in 
American Correctional Education 

Special Populations — Rural/l "rhan 

Selected Vocational Preparation Resources for Serving 
Rural Youth and Adults with Special Needs 



Selected Vacationed Preparation Resources for Serving 
Urban Youth and Adults with Special Needs 

Special Populations — Exemplary Practice 

Effective Vocational Education for Students with 
Special Needs: A Framework 

Special Populations— LKP 

Collaboration for Instruction ofLEP Students in 

Vocational Education 
Preparing Adult Immigrants for Work: The Educational 

Response in Two Communities 

Special Populations — Teen Parents 

Access to and Use of Vocational Education in Teen 
Parent Programs 

Gender Equity: A Resource List of Organizations and 
Information Centers (MDS-744CA) was compiled b\ the 
Office of Special Populations ai the NCRVE site at the 
University of Illinois. Exemplary Programs Serving 
Special Populations. Volume I (MDS-303CA) was 
prepared by Zipura T. Burac and Exemplary Programs 
Serving Special Populations, Volume 2 (MDS-424CA) 
was prepared by Zipura T. Burac and Robert Yanello, 
hoih from the Technical Assistance for Special 
Populations Program (TASPP) at the NCRVE site at the 
University of Illinois. (For ordering information, see page 
12.) 



REVISING EDUCATIONAL ASSESSMENT 
Accountability & Assessment 



Like any successful educational initiative or set of 
initiatives, the school-to-work initiative will recjuhe 
effective (and perhaps unique) methods of assessment 
and systems of accountability. Like the many 
connecting activities alluded to above, issues of 
accountability and assessment arc not unique to school- 
to-work initiatives, but such issues must necessariU be 
discussed within the context of school to- work. 

Two items about accountability and assessment 
available from NCRVU arc reviewed here- -the first is a 
monograph, the second a videotape. 



In Assessment in Education: A Search for Clarity in the 
Growing Debate, author John Lulling attempts to clarif y 
recent developments in educational assessment. As 
anyone involved with or following educational reform 
knows, summarizing the developments in any area of 
education over the course of the last decade is no simple 
task — no less so in an area as contttnersial as 
educational assessment. Por educators and 
decisionmakers. Assessment in Education senes as a 
concise and valuable guide to the more significant 
developments in assessment over the last decade. 
Whether they be reforms inspired b\ the need to 
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develop new forms of vocational education in response 
lo changing economic and demographic conditions, 
reforms mandated by legislation such as the 1990 
Amendments lo the Carl D. Perkins Vocational 
Education Act, or reforms driven by the persistent 
criticisms of American education in general, 
developments in educational assessment during the last 
ten or so years have fortunately been numerous and 
diverse — so numerous and diverse, in fact, as to be 
overwhelming to those making decisions about 
assessment for their classrooms or their schools. 

Assessment in Education is not so much about the 
specific assessment reform initiatives that have 
developed over the last decade as it is about the 

Interest is growing in 

alternatives to pencil-and- 
paper tests that assess not 
just what students know, 
but how they can apply 




foundations upon which the philosophies of assessment 
are formed. Throughout his monograph, Latting refers 
to and speaks in terms of three "camps'* of educational 
assessment: The first, [he psychometric tradition* 
"treat) s| the measurement of both aptitude and 
achievement as an exact science" and "assign) s| 
quantities to a whole range of mental properties." The 
second and third, performance assessment and 
alternative assessment, are both reactions against the 
psychometric tradition, the forms of which (most 
notabl), multiple choice tests) are criticized for, among 
other things, their cultural bias, their inability to test for 
what interests educators, and their "deleterious" effects 
on instruction and learning. The second camp. 
performance assessment, emphasizes "the supposed 
failure of tests lo measure accurately the kind of 
know ledge that is of most use to society"; the third. 
alternative assessment, emphasizes the instructional and 
learning effects of testing. 

Alter defining these camps and he I ore discussing the 
various fuims i>l assessment envisioned bv each. 
Assessment m hlucation piesents the characteristics of 
assessment in general, halting begins w ith the uses of 



educational assessment: classroom instructional 
guidance; system monitoring; and selection, placement, 
and certification of students — clarifying and giving 
examples of each. Next, the author defines and describes 
what he calls the organizational, physical, and technical 
features of educational assessment. Any assessment, 
Latting contends, is a construction of several formal 
features or variables that define the nature of that 
particular assessment. Finally in discussing the 
characterisf.es of assessment, Assessment in Education 
broaches the notions of reliability — the consistency of 
test scores — and validity — the extent to which 
inferences based on a test score are appropriate. 

The bulk of Assessment in Education is devoted to 
detailed discussions and evaluations of the three 
distinct, though not mutually exclusive assessment 
camps. Most of the discussion and evaluation takes the 
form of summaries of the literature concerning 
assessment. Each of the camps is taken up in order, with 
the discussions of performance assessment and 
alternative assessment following the discussion of the 
psychometric tradition and presented as much in 
reaction to the tradition as camps in their own right. In 
an effort at objectivity, however, Latting includes a 
section entitled "A Response from the Psychometric 
Tradition" to round out the discussions in the section. 

The value of Assessment in Education as a guide to 
those interested in understanding assessment is qualified 
by one inherent though expressed limitation. That 
limitation, however, points back to the value of the 
guide, according to W. Norton Grubb, NCRVH Site 
Director at the University of California at Berkeley and 
author of Assessment in Education** preface: 

Unfortunately — as is often the case in periods of 
reform — there are not yet clear results. The new 
assessments described in this monograph are lor 
the most part Mill far from completion. None of 
them is going to be able to serve all the purposes 
that tests now serve. This makes it all the more 
important, however, for educators to understand 
the nature of current developments so that they 
can distinguish those approaches to assessment 
that are likely to be useful for their specific 
purposes from others that are unlikely to be of 
much help. 

Assessment in Education should be of help to educators 
in this respect. 
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The hour-long videotape Assessment 2000: A a 
Exhibition is the product of a 1993 interactive video 
teleconference about three forms of stude.it assessment 
used as alternatives to traditional pencil-and-paper 
testing: the student project, the performance event, and 
portfolios. Teleconference host and NCRVE researcher 
E. Gareth Hoachiander discusses these assessment 
alternatives with a guest panel that includes a student 
and representatives of education; business; and staff of 
Far West Laboratory, co-producers of the videotapes. 

Innovators are constantly 
under pressure to justify 
the success of their 
reforms, to show 
improved outcomes, and 
to prove to legislators that 
their funds are well spent 

The guests were assembled for the teleconference to 
describe the three assessment techniques and to share 
their experiences in using the techniques. In addition, 
throughout the course of the program, Hoachiander and 
his guests field questions I mm teleconference 
participants calling in from various remote sites. 

Production of the teleconference, according to 
Hoachiander, was inspired by growing interest in 
alternatives to pencil-and-paper tests that assess not just 
what students know, but how they can apply that 
knowledge. 

The student project technique is covered first. While the 
requirements for such projects var\, the) generally 
include the following components; 

a research paper 

a tangible product 

an oral exhibition 

Student projects also ha\c tour ke> leatiues. 

The) arc appropriate for all students 



0 They involve writing, doing, and speaking 

* They require extended commitment on the pail of 
students 

* They foster student-teacher interaction 

In addition to several other issues, the ways in which 
student projects improve assessment and caveats of the 
technique are highlighted. 

The performance event requires students to apply their 
knowledge and skills in work-related situations, either 
real or simulated. In performance events, students are 
required to 

diagnose problems 

identify alternative solutions 

;ake appropriate actions 

successfully complete tasks 

Discussion during the teleconference focuses on the 
nature and advantages of the performance event 
technique and on important considerations of 
performance event planning. 

The last of the three techniques covered, the portfolio, is 
the most commonly implemented of the three and 
therefore is very likely familiar to those viewing the 
teleconference. Because the use of portfolios as 
assessment tools is fairly common, the components and 
features of implementation \ary widely. As with the 
other two methods, however, the advantages and other 
important considerations of the method are discussed. 

Assessment 2000 seives as a good introduction to the 
three assessment techniques. In addition to providing 
basic information about the nature of each technique and 
the important considerations that go into implementing 
them, the teleconference attempts to frame the 
techniques w ithin the scope of more traditional testing 
methods. 

.\sm wmciU in l.tluctition: . \ Search for CUnit\ in tin 
Cf>cm//i# Debutv (MDS-254(/Ai was piepared In John 
1 atting I rum the NCRYK site .it the I'niversit) ol 
California at Berkeley and Assessment 2000: .\n 
Inhibition (NinS-76<SCA> w.is produced In NCRVI* and 
hu West Laboratory (loi ordering mlormalion. see page 12 i 
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ABOUT NCRVE 



The National Center for Research in Vocational 
Education (NCRVE) is established under authorization 
of the Carl D. Perkins Vocational Education Act to 
engage in two vocational education-related activities: 
(1) applied research and development and (2) 
dissemination and training. Our mission is to address 
the goals of preparing individuals for substantial and 
rewarding employment and of acting as a catalyst for a 
shift to an economy dominated by a skilled and flexible 
workforce. 

NCRVE is a consortium of educational institutions led 
by the University of California at Berkeley's Graduate 
School of Education. This consortium also consists of 
the Teachers College at Columbia University, the 
University of Illinois, the University of Minnesota, RAND, 
Virginia Polytechnic Institute and State University, and 
the University of Wisconsin at Madison. 

In addition to its research and development agenda, 
NCRVE supports five dissemination and training 
programs: (1) dissemination; (2) professional 
development; (3) special populations; (4) planning, 
evaluation, and accountability; and (5) program 
development, curriculum, and instructional materials. 
This publication is part of our commitment to those on 
the front lines of vocational education, striving to create 
a new vision. 

NCRVE Administrative Staff 

Phyllis Hudecki Acting Director 
Olivia Moore Deputy Director 
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are provided by the NCRVE Materials Distribution 
Service. For subscriptions and address changes, contact 
the Materials Distribution Service (see page 12). In line 
with our goals, Change Agent is free of any copyright 
restrictions and may be copied or quoted freely. 

We welcome your suggestions, criticisms, and 
questions. Contact Peter Seidman, Dissemination 
Program Director, NCRVE, University of California at 
Berkeley, 21 50 Shattuck Avenue, Suite 1 250, Berkeley, 
CA 94704. Phone: (800) 762-4093 or (510) 642-4004; 
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THE RIGHT INSTRUMENT 
Measuring Leadership Development 



t wo recently revised instruments, developed at the 
I NCRVE site at the University of Minnesota, 
JL provide users with tools for assessing the 
leadership performance of individuals in vocational 
education. As their titles indicate, the guides for the two 
instruments — Leader Effectiveness Index Manual and 
Leader Attributes Inventory Manual, both by Jerome 
Moss, Jr., Judith J. Lambrecht, and Qetler Jensrud from 
the University of Minnesota site and Curtis R. Finch 
from the Virginia Polytechnic Institute and State 
University site — outline the development and use of the 
instruments. 

"Education is a latecomer to the study of leadership and 
almost no research has been done in vocational 
education," according to the Leader Attributes Inventory 
(LAI) Manual. As yet, "there is no consensus on a 
specific definition of leadership, an explanatory model 
of leadership behaviors, or the most useful means for 
measuring the effectiveness of leaders. There is, 
however, substantial agreement that leadership behavior 
can be measured and shown to be related to effective 
performance and that educational interventions can 
effect the behavior of leaders." Both the LAI and the 
Leader Effectiveness Index (LEI) are efforts *n the 
direction of that agreement. 

NCRVE's role in leadership development is not simply 
mandated by legislation (Perkins legislation, 
specifically). Because NCRVE exists as a national 
leader in vocational education, its role is the natural 
outgrowth of a nationwide demand for effective 
leadership development. The need for strong leaden; 
(and consequently, for effective leadership 
development) is also an inherent result of unstable 
situations like that in which vocational education finds 
itself — unstable because of the changes that are rapidly 
and significantly impacting education and the social and 
economic environment in which it exists. NCRVE's 
goal in the area of leadership development is to foster 
the number and quality of leaders prepared to meet 
present and future challenges of vocational education. 

Tiie LAI yields a diagnostic assessment by multiple 
observers (including the individual being assessed) of 
37 attributes — characteristics, knowledge, skills, and 
values — that predispose individuals to desirable 
leadership performance in vocational education. The 
manual for the inventory gives the rationale for the LAL 
describes the developmental and psychometric 



characteristics assessed by the inventory, and explains 
the inventory's use. The manual is designee especially 
for potential users of the inventory but would also be of 
interest to those who study leadership and leadership 
measurement. 

Comprehensive assessment of an individual with the 
LAI involves a self- and observer ratings. For both types 
of ratings, 37 items assess the 37 attributes with a 6- 
point response scale that describes the extent to which 
the person being rated possesses each attribute. Among 
the attributes are insightiulness, adaptability, 
accountability, dependability, communication, ability to 
delegate, and ability to manage information. The 
individualized feedback ieport produced from the 
inventory contains three charts, which ( 1 ) compare a 
ratee's self-ratings with the average rating of his or her 
observers, (2) compare the average rating of his or her 
observers for each attribute with those of an appropriate 

The LAI assesses 
attributes that predispose 
individuals to desirable 
leadership performance 
in vocational education 

norm group, and (3) predict the level of leadership 
performance expected of the participant in his or her 
norm group. 

Because the ideas of norms and standards are so 
important, "second in importance only to the 
identification of relevant attributes and their consistent 
measurement," the authors of the LAI Manual go to 
lengths to describe the extensive process used to 
establish the inventory's norms and standards. In 
addition, the development of the instrument as a whole 
is explored along with its reliability and validity. 

There are two major reasons lor using the LAI: ( 1 ) 
assessment of leader attributes at a point-in-time and (2) 
measurement of change in leader attributes for an 
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individual over time. Self-assessment is only the first 
step in point-in-time leader assessment. Self- 
assessments focus an individual's attention on the 
attributes that predispose one to desirable leader 
performance and establish developmental goals by 
pinpointing the leader attributes to be strengthened. 
Subsequent assessments by three to five subordinates 
(preferably; or peers then highlight discrepancies in 
perception. Knowledge of such discrepancies help rateto 
understand frictions in interpersonal relationships and 
motivate them to strengthen selected attributes, on the 
one hand, and help build self-confidence with improved 
performance, on the other. 

Measuring change in leader attributes with the LAL is 
accomplished with multiple self-assessments only: one 
before a leadership development program and two 
after — one in which ratees rate themselves as they now 
perceive themselves to have been before beginning a 
leadership program such as NCRVE's Preparing 
Leaders for the Future curriculum (see "For Those 
About to Lead" on pages 6-8) and one (taken a few 
minutes after the first) in which ratees rate themselves 
as they currently perceive their attributes. 

In addition to a copy of the LAL rating-by-observer form, 
a sample of an individualized feedback report, and raw- 
to-normalized-score conversion tables, the LAI Manual 
includes a copy of the Leader Effectiveness index (LEI) 
and information about its development. The LEL is 
included because the authors contend that it "provides 
an important criterion measure used to estimate the 
validity of the LAI" 

Explored in the Leader Effectiveness Index Manual, the 
LEI instrument itself is much shorter than the LAI. The 
LEI consists of seven items that assess the extent to 
which vocational education leaders facilitate group 
processes aud empower group members generally and 
the extent to which they perform six broad tasks: 

1. inspire a shared vision and establish standards that 
help the organization develop 

2. foster unity, collaboration, and ownership as well as 
recognize individual and team contributions 

3. exercise power effectively 

4. exert influence outside of the organization in order 
to set the right context for the organization 

5. establish an environment conducive to learning 



6. satisfy the job-related needs of members of the 
organization as individuals 

As in the LAI, attributes in the LEL are rated for a.* 
individual with a 6-point response scale, from "not 
effective" to "extremely effective;' However, while the 
LA! provides a diagnostic assessment of the attributes 
that predispose an individual to desirable leader 
performance, the LEL measures the effectiveness of 



Leadership behavior 
can be measured and 
shown to be related 
to effective performance 



leader performance. Also like the LAI, the LEI can be 
used either for point-in-time assessment or for 
measuring changes in leader performance. 

The LEI instrument can be completed in less than five 
minutes, and it is suggested, as with the LAI, that three 
to five subordinates or peers who know the ratee well be 
used as observer-raters. Unlike the LAI, including 
information about raters (e.g., gender, ethnic group, 
position) with the instrument is optional. 

The LEL is supported by the same kind of research and 
rigorous development process that contributed to the 
development of the LAI. Information about the research 
and development process are included in the manual 
with special attention given to the process of 
establishing norms and standards for the index. A copy 
of the LEL instrument is appended with a sample 
individualized feedback report. 

Leader Attributes Inventory Manual (MDS-730CA) and 
Leader Effectiveness Index Manual (MDS-8 15CA) were 
prepared by Jerome Moss, Jr., Judith J. Lambrecht, and 
Qetler Jensrud from the NCRVE site at the University of 
Minnesota and Curtis R. Finch from the NCRVE site at 
Virginia Polytechnic Institute and State University. (For 
ordering information, see page 12.) 
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ADMINISTRATIVE BREAKWATER 
A Professional Development Simulation 



Curtis R. Finch's Breakers: An Organizational 
Simulation for Vocational Education 
Professionals is unique among NCRVE 
publications. It contains all the materials users need to 
conduct an organizational simulation. In the simulation, 
participants play the roles of administrators (e.g., vice 
presidents and deans) responsible for the operation of 
Breakers Technical College, a postsecondary institution 
serving 10,000 full- and part-time students. 

The Breakers materials include everything short of 
common office supplies required to conduct the nine- to 
twelve-hour simulation: guidelines for conducting the 
simulation, Breakers Technical College orientation 
materials, and materials for each of the ten 
administrative roles. Included in the guidelines are 
checklists and forms intended to assist those directing 
this highly interactive simulation, which allows 
participants to apply leadership knowledge and skills in 
various situations. Multiple simulations can be run 
simultaneously with one simulation director and three 
director's assistants per simulation along with enough 
individuals to fill the ten administrative roles. 

The simulation, which was by accounts well-received in 
field tests, builds on NCRVE leadership research; 
personal interviews with vocational administrators from 
thirty-eight United States secondary and postsecondary 

The Breakers simulation 
allows participants to 

apply leadership 
knowledge and skills 
in various situations 

institutions; and additional interviews with groups of 
technical college, community college, and technical 
center administrators and instructors. 

The "Conducting the Simulation" section of Breakers 
provides a brief outline of the simulation's organization 
and operation as well as checklists for before, during, 
and after the simulation; a list of materials; a guide for 



the simulation debriefing; samples of documents for 
administering the simulation; and information about the 
origination, distribution, and focus of the simulation- 
driving coi;°spondence. The ^Orientation Materials" 
section includes a sample simulation itinerary and 
information about Breakers Technical College. 

The bulk of Breakers consists of correspondence — 
correspondence from within and from outside the 
college to administrators within the college who are the 

Well-received in field tests, 
Breakers builds on NCRVE 
leadership research 

key players in the simulation. In addition to copies of 
the correspondence that the respective administrators 
themselves have "sent" to others, administrators begin 
with copies of correspondence "sent" by other 
administrators in the theoretical lime before the actual 
events of the simulations have taken place. 

The situations that arise during the course of the 
simulation and with which participants must deal relate 
to special events; report findings; enrollment, student, 
curriculum, personnel, facilities, and program concerns; 
and public relations matters. The situations are 
introduced in the correspondence provided with the 
simulation materials and are addressed by participants 
during the course of the simulation. 

Potential users of the Breakers materials, especially 
those who plan on running concurrent simulations, 
should be aware that Breakers — like all NCRVE 
publications — is in the public domain and is therefore 
free of copyright restrictions. Multiple copies of the 
materials may be made at the users discretion as long as 
NCRVE is acknowledged as their source. 

Breakers: An Organizational Simulation for Vocational 
Education Professionals (MDS-278CA) was prepared by 
Curtis R. Finch from the NCRVE site at Virginia 
Polytechnic Institute and State University. (For ordering 
information, see page 12.) 
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Case Studies for Leadership Development 



The collection of case studies included in Case 
Studies in Vocational Education Administration: 
Leadership in Action are intended to "fill a 
void/* According to the authors of the collection — 
Curtis R. Finch, Cecil E. Reneau, Susan Faulkner, 
James A. Gregson, Victor Hernandez-Gantes, and Gayle 
A. Linkous, all of NCRVE's site at Virginia Polytechnic 
Institute and State University — case study textbooks for 
educational professionals in general and for vocational 
and technical education administrators in particular are 
rare. Yet, "Case studies have long been used by business 
and industry to cultivate and nurture critical leadership 
skills in key management personnel," according to the 
authors. "Similarly, other professions — in both public 



These case studies are an 
extension of NCRVE 

leadership research and 
reflect the results of 
extensive field testing 



and private sectors of the workforce — have recognized 
the value of case studies and begun using them to 
facilitate the transfer of conceptual knowledge and 
understanding to situational applications/* 

The lack of "illustrative and illuminating" case studies 
focusing on the issues currently confronting vocational 
and technical education is the primary inspiration for 
Case Studies. "Such issues, indigenous to the vocational 
education profession" the authors contend, "cannot be 
adequately addressed by case studies written for other 
career groups." 

The fifty -one case studies included in the collection are 
drawn from and build upon the results of the NCRVE 
publication entitled Leadership Behaviors of Successful 
Vocational Education Administrators (MDS-097CA, 
see page 12 for ordering information) and reflect the 
broad scope and complexity of organizational dynamics 
in vocational education. Collectively, the case studies 
are an educational resource thoughtfully designed to 
assist aspiring and practicing vocational education 
leaders "in achieving their greatest potential." Not only 



are the case studies an extension of vocational education 
leadership research, but they also reflect the results of an 
extensive, six-university review and field testing 
process. 

The cases are organized into twelve functional groups, 
an arrangement that classifies the cases by common 
themes. While each case is assigned to only a single 
group, a number of them can be applied in more than 
one of the twelve areas. Below is a brief outline of the 
groups: 

Group 1: Determining Institutional Direction, Goals, 
and Policies 

Group 2: Participating in Accreditations and 
Evaluations 

Group 3: Collaborating with Boards, Agencies, and 
Organizations 

Group 4: Implementing Mandated Changes and 
Improvements 

Group 5: Implementing Self-Selected Changes and 
Improvements 

Group 6: Linking with Business, Industry, and the 
Community 

Group 7: Enhancing Institutional Visibility and Image 

Group 8: Maintaining and Improving Fiscal Posture 

Group 9: Securing and Improving Facilities and 
Equipment 

Group 10: Handling Crises 

Group 1 1 : Resolving Staff and Student Problems 

Group 12: Participating in Individual and Group 
Discussions 

The first two cases in each group are complete stories, 
describing an event or situation as it evolves. Thus, 
users see how a situation developed and how it was 
dealt with by an administrator. These cases more readily 
lend themselves to retrospective analyses. The 
remaining cases in each grouo are open-ended, bringing 
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readers to a point in a situation where they must 
consider what went wrong or what difficulty arose then 
decide what to do next. "These cases/' according to the 
authors, "are most suitable for developing problem- 
solving and decision-making skills." 

Each case study is two to four pages in length and 
includes a set of discussion questions that prompt 
participants to examine closely and think beyond the 
specific circumstances of the cases as presented. 

The cases may be used in a variety of learning 
environments — from university classrooms to 
professional development workshops to institutional 



meetings. The introductory material of Case Studies 
provides some additional information about the logistics 
and pedagogical specifics of using the case studies for 
maximum effectiveness. 

Case Studies in Vocational Education Administration: 
Leadership in Action (MDS-279CA) was prepared by 
Curtis R. Finch, Cecil E. Reneau. Susan Faulkner, James 
A. Gregson, Victor Hernandez-Gantes, and Gayle A. 
Linkous from the NCRVE site at Virginia Polytechnic 
Institute and State University. (For ordering information, 
see page 12.) 



FOR THOSE ABOUT TO LEAD 

A Leadership Development Curriculum 



The focus of Preparing Leaders for the Future: A 
Development Program for Under represented 
Groups in Vocational Education by Jerome 
Moss, Jr., Sherry Schwartz, and Qetler Jensrud is to 
enhance, adapt, and cultivate leader attributes for 
effective performance in vocational education 
leadership roles. It is a response, as are the other 
NCRVE products reviewed in this issue of Change 
Agent, to a consensus among vocational educators that 
there is currently a pressing need for more high quality 
leaders and that the field has not, until recently, made an 
adequate effort to prepare them. 

Recognizing the need for effective vocational education 
leaders, the NCRVE seeks to stimulate, facilitate, and 
evaluate educational interventions that effect positive 
change in selected leader attributes among potential and 

The Preparing Leaders 
for the Future program 
will prepare participants 

to seek out and take 
advantage of leadership 
opportunities 



practicing leaders. The purpose of the Preparing 
Leaders for the Future program is to provide leadership 
development experiences specifically for women and 
other underrepresented groups in vocational education 
and to increase the likelihood that individuals in such 
groups will perceive opportunities to assume leadership 
roles, grasp those opportunities, and succeed as leaders 
in a wide range of situations and professional positions. 

Given the strong relationship between the leader 
attributes assessed in NCRVE* s Leader Attributes 
Inventory (see 'The Right Instrument" on pages 9-10) 
and desirable performance as a leader and given the fact 
that many attributes can be significantly improved 
through formal instruction, the enhancement, adaptation, 
and cultivation of the leader attributes has been adopted 
as the instructional task and primary content base of the 
Preparing Leaders for the Future program, which is in 
fact a leadership development curriculum. Because most 
underrepresented groups have had limited opportunities 
to gain experience in leadership roles, the curriculum's 
designers decided that the curriculum should provide 
learning experiences in a wide range of leader attributes. 
Opportunities are provided for individuals to attain 
depth in those attributes they select, rather than 
experiences in a few predetermined attributes. 

Prior NCRVE research on leadership programs for 
graduate students has shown that there is a high positive 
correlation between hours of directly supervised 
instruction and the number of leader attributes improved 
and that 90 hours of classroom instruction (the length of 
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the Preparing Leader for the Future curriculum) plus 
outside assignments is adequate to significantly affect a 
meaningful number of leader attributes and skills. 
However, the designers contend, even with 90 hours of 
class instruction, it is not reasonable fur a program such 
as Preparing Leaders for the Future to attempt to 
improve all leader attributes of all participants. The 

There is a high positive 
correlation between hours 
of directly supervised 
instruction and the 
number of leader 
attributes improved 

designers of the curriculum, therefore, found it 
necessary to identify the particular attributes whose 
improvement would constitute the specific instructional 
objectives of the program. 

Two sets of attributes were chosen. One set consists of 
fourteen attributes that best explain the variance in 
leader performance and/or that were consistently 
considered by former participants of leadership 
programs to have been most useful to them in their 
leadership activities. Included among the fourteen 
attributes are insightfulness, ability to motivate others, 
ability to make decisions, and ethicality. 

As for the second set. although all successful leaders 
should demonstrate common attributes, individual 
differences tend to create particular strengths as well as 
developmental needs. Eleven attributes, the further 
development of which is likely to be helpful to most 
underrepresented groups, were identified using a 
literature review and the advice of individuals from 
underrepresented groups who have had experience 
conducting leadership development programs. Those 
attributes include tolerance of ambiguity and 
complexity, ability to manage conflict, tolerance of 
frustration, and ability to use appropriate leadership 
styles. These attributes, incidentally, also form the 
foundation for the objectives of the program. 

The Preparing Leaders for the Future program is 
divided into three major sections: ( I ) "Introduction to 
the Leadership Development Program," (2) "Planning 
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for Leadership Development," and (3) "Developing 
Leader Attributes/' The "Introduction" section contains 
an orientation to the program and its processes, a view 
(from various perspectives) of vocational education 
system realities, and an introduction to group process 
skills used throughout the program. The "Planning for 
Leadership Development" section of the program 
includes an assessment phase (of life roles, personality 
type, and leader attributes) and a goal- and strategy- 
development phase (using mentors and formulating an 
individualized leadership development plan). 

The first two sections set the stage for moving into the 
third and largest section of the curriculum, "Developing 
Leader Attributes." In developing the learning 
experiences of the program for the attributes selected, 
the program designers report, it became evident that a 
three-category scheme of leader attributes made 
pedagogical sense. Obvious interrelationships and 
mutual reinforcement among the attributes within each 
category were apparent. The major program section of 
"Developing Leader Attributes" is, therefore, organized 
into three phases: (1) personal characteristics, (2) 
interpersonal skills and characteristics, and (3) 
management knowledge and skills. 

The Preparing Leaders for the Future program is 
intended to be customized for specific contexts and 
constituents. The various potential uses for it, according 

The Preparing Leaders 
for the Future program 
will begin to provide 
participants with the 
knowledge and leadership 
skills necessary for 
effective performance 

to its designers, imply the need for a flexible program 
structure in which each learning experience is focused 
on only one or two attributes and is relatively 
independent from other learning experiences. At the 
same time, the designers recognize that the most 
effective instruction (pedagogically speaking) is 
probably achieved when learning experiences take 
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advantage of the inherent interrelationships among 
attributes and the opportunities they may present for 
mutual reinforcement. According to the designers, 
therefore, 

The structure of the program tries to strike a 
reasonable balance between these alternatives. 
Each learning experience focuses on specific 
attributes so that it is possible to select particular 
learning experiences to achieve targeted 
outcomes. At the same time, learning 
experiences also try to make interrelationships 
among attributes evident where they exist, and 
assignments often try to provide practice on 
more than one attribute. 

The nature of the program's content, with its intent to 
change personal characteristics and interpersonal skills, 
lends itself to the use of group-instruction techniques to 
create and take advantage of supportive groups of 
participants. Consequently, learning experiences are 
structured by the program designers for delivering 
common content to a group of participants. In order to 
provide for the unique needs of individual 
participants — made evident in the assessment and 
planning phases of the program — participants have been 
allowed to individualize assignments, and they are 
expected to undertake special long-term projects. 

The structure of each learning experience in Preparing 
Leaders for the Future ensures that instruction focuses 
on changing performance. The program was designed 
with an eye to the practical matters of delivering 
curriculum. It is this specificity and practicality that 
make this two-volume program so valuable — and, to a 
certain extent, unique. Participants do not simply learn 



NCRVE seeks 
to stimulate, facilitate, 
and evaluate educational 
interventions that effect 
positive change in selected 
leader attributes 



are presented in behavioral terms and serve as the 
objectives of the various learning experiences. Each 
behavioral objective is then analyzed as a means for 
identifying the content to be taught. A lesson delivery 
plan is then created from the analysis to guide the 
instructor and maintain participants' focus on the 
respective objectives of instruction. Each lesson 
delivery plan then follows a set of common steps: 
content, method, and media are recommended for each 



There is currently a 
pressing need for more 
high-quality leaders 



step; for practice, assignments (some, iong-term 
assignments) are provided that allow participants to 
demonstrate desired leadership behaviors. The materials 
for each learning experience also provide an estimate of 
the in-class ftod out-of-class time required to complete 
the experience, a list of necessary and optional 
instructional materials, and a list of optional activities. 

In contrast — and in response — to these views, the 
Preparing Leaders for the Future program utilizes the 
assets of underrepresented groups in vocational 
education leadership roles to create positive change: 

It is not proposed that this program will 
eliminate the special challenges that 
underrepresented groups may encounter in 
attaining leadership positions due to 
discriminating practices and attitudes. It will, 
hov.ever. prepare participants to seek out and 
take advantage of leadership opportunities and 
will begin to provide participants with the 
knowledge and leadership skills necessary for 
effective performance. 

Preparing Leaders for the Future: A Development 
Program for Underrepresented Groups in Vocational 
Education (MDS-736CA) was prepared by Jerome Moss. 
Jr.. Sherry Schwartz, and Qetler Jensrud from the 
NCRVE site at the University of Minnesota. (For 
ordering information, see page 12.) 



about attributes; they are required to demonstrate 
respective behaviors. The definitions of the attributes 
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A SOURCE OF RESOURCES 

Selecting & Applying Resources for Leadership Development 



Included among the myriad other urgencies that 
sprouted from educational reform efforts of the last 
decade or so is the urgency to improve the 
preparation of educational leaders. Vocational 
Education Leadership Development Resources: 
Selection and Application by Curti:; R, Finch. James A. 
Gregson. and Cecil E. Reneau — citing reports such as 
the National Commission on Excellence in Education's 
Leaders for America \s Schools, which calls for a 

Vocational Education 
Leadership Development 
Resources provides 
detailed information 
about leadership 
development resources to 
those responsible for 
developing vocational 
education leaders 



redefinition of educational leadership; the American 
Association of Colleges for Teacher Education's School 
Leadership Preparation, which presents a series of 
recommendations for reconceptualizing administrator 
preparation programs; and the National Policy Board for 
Educational Administration's Improving the 
Preparation of School Administrators — responds to the 
urgency by giving detailed information about leadership 
development resources to those responsible for 
developing vocational education leaders. 

Such resources, necessarily designed to facilitate the 
development of leader attributes, "can often spell the 
difference between a substandard and a successful 
program," contends this NCRVE guide. As leadership 
programs are established 01 modified to meet the 
challenges posed by changes in education and as serious 
consideration is given to the organization, content, and 
delivery of these programs, attention must eventually 
and inevitably be paid to the selection and use of 



leadership development resources. When the attention 
of those responsible for developing vocational education 
leaders does focus on this important and ongoing aspect 
of leadership development, metaresources like 
Vocational Education Leadership Development 
Resources serve useful and practical roles. 

Vocational Education Leadership Development 
Resources begins by developing a context for leadership 
development resources. It outlines a variety of 
"arrangements" — graduate degree programs; leadership 
academies; institution, organization, and agency 
programs; certification arrangements; leadership 
seminars: fellowships; and assessment centers — and 
addresses briefly the concerns faced by leadership 
development programs. Then, after establishing a 
foundation with respect to where resources like those 
listed in the guide might be used in the vocational 
education leadership development process, the guide 
discusses how the quality resources necessary can be 
selected. 

The foundation that the authors establish for their 
collection is rooted in a framework that emphasizes the 
role of teamwork and is driven by a five-question 
planning paradigm of the leadership development 
process. This framework links leadership development 

Good resources often spell 
the difference between a 
substandard and a 
successful program 



with the resources that the development process utilizes, 
a process that is characterized by four phases: 

1. The foundation phase, which emphasizes 
development of knowledge 

2. The bridging phase, which "bridges" the gap 
between foundational studies and field experience 

3. The practicum phase, characterized by structured 
and monitored experiences in actual educational 
settings 
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4. The practice phase, in which individuals "learn on 
the job" 

Each phase is typically associated with a specific set of 
resource types. These types are defined in Vocational 
Education Leadership Development Resources and 
addressed in terms of how they can be located: 

textbooks 

handbooks 

lectures and discussions 

• self-assessments 

• simulations 
° case studies 

• games 

• shadowing 

• interning 

• extern ing 
° mentoring 

• induction 

One of the two strengths of this publication are its 
valuable appendices, which include an annotated 
bibliography of leadership development resources and 
services, list of leadership development providers, and 
collection of leadership resources profiles. 

The other strength of the guide is the "Assessment 
Form " developed by the authors with the assistance of a 
national panel of leadership development experts to 
uniformly assess leadership development resources of 
varying formats, emphases, and pedagogies. 
Specifically, the form is designed to assess the 
following with respect to a resource: 

o organizational structure 

o potential audience 



instructional techniques employed 
media requirements 

phases of leadership development addressed 

• its research base 

• specific applicability to vocational education 
leadership development 

perceived cost-effectiveness 

strengths and limitarions 

For those in the process of developing or improving a 
vocational education leadership development program. 

In the selection and use 
of leadership 

development resources, 
metaresources like this 
one serve useful and 




the form itself — let alone its supporting material — is 
worth the publication's cost-recovery price. Vocational 
Education Leadership Development Resources defies 
unstructured, unarticulated, hit-and-miss resource 
selection. 

Vocational Education Leadership Development 
Resources: Selection and Application (MDS-188CA) was 
prepared by Curtis R. Finch, James A. Gregson, and 
Cecil E. Reneau from the NCRVE site at Virginia 
Polytechnic Institute and State University. (For ordering 
information, see page 12.) 



• leader attributes addressed 

instructional settings in which it can be used 
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Change Agent '9 

on 't Miss a Single Issue! 

The 1995 volume of Change Agent will highlight research in some of 
vocational education's hottest issues — School-to- Work, All Aspects of the 
Industry, and much, much more. 

And, as always, each issue of Change Agent . . . 

Saves you time by extracting, evaluating, and formatting information for 
quick review 

Includes key findings for frontline practitioners and policymakers 

Keeps you current with successful practices nationwide 

Is authoritative — NCRVE research is supported by the Office of 
Vocational and Adult Education, U.S. Department of Education 

Change Agent 

Shaping the Future of Vocational Education 



Change Agent Order Form 



□ Yes! Please send me the 4 issues of Change Agent '95. I understand that at the low subscription 
rate, NCRVE cannot afford to bill me. . . . Enclosed is my check for $25, made payable to NCRVE 
Materials Distribution Service. 



Send my issues of Change Agent to 



Name 
Address 



Title 



City / State / Zip 

Telephone _ 

Questions about orders should be directed to 
ni -$- Burnell at (800) 637-7652. 
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Breakers: An Organizational Simulation for 
Vocational Education Professionals 
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Case Studies in Vocational Education 
Administration: Leadership in Action 
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To order items reviewed in Change Agent, please 
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NCRVE Materials Distribution Service 
Western Illinois University 
Horrabin Hall 46 
900 W. Adams Street 
Macomb, IL 61455-1396 
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Phone: (800) 637-7652 
Fax: (309) 298-2869 
E-mail: msmds@bgu.edu 



NCRVE materials are distributed on a nonprofit, 
cost-recovery basis. All NCRVE publications 
are in the public domain and therefore free of 
copyright restrictions. Multiple copies of the 
materials may be made at the users discretion 
as long as NCRVE is acknowledged as their 
source. 
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